About the role
Job Description:As a Quality Assurance Engineer at Gurully, you will play a crucial role in ensuring the reliability, performance, and security of our AI-driven learning platforms used by students and institutes worldwide. The ideal candidate will have a minimum of 3 years of hands-on experience in QA, with a proven track record of building test strategies, automating tests, and validating AI-powered features. The role involves working closely with product and development teams to identify issues early, drive continuous improvement, and deliver a seamless experience to our users.
Responsibilities:
- Review and analyze requirements, specifications, and technical design documents, providing timely feedback.
- Plan, create, and manage the overall quality strategy, including detailed test plans and test cases.
- Design, develop, and execute automated tests, along with manual, UI, and regression testing.
- Test AI-powered features such as automated scoring and feedback for speaking and writing tasks, validating accuracy, consistency, and fairness against expected benchmarks.
- Evaluate AI/LLM outputs for hallucinations, bias, edge cases, and response quality, and build reusable test datasets for ongoing validation.
- Leverage AI-assisted tools to accelerate test case generation, test automation, and defect analysis.
- Test current products, identify and report defects, and verify that issues are resolved.
- Investigate the root causes of quality issues and suggest practical solutions to improve customer satisfaction.
- Define and track key quality KPIs such as defect density, open defect counts, and AI scoring accuracy.
- Identify bottlenecks in the QA process and drive continuous improvement initiatives.
- Collaborate with the product and development teams to ensure consistent, on-time project delivery.
- Stay updated with new testing tools, AI testing techniques, and industry best practices.
- Prepare and present quality reports and metrics to senior management.
- Bachelor's or Master's degree in Computer Science or a related field.
- Minimum of 3 years of proven experience as a Quality Assurance Engineer.
- Hands-on experience with white box and black box testing, UI testing, and automated testing.
- Proficient in automated testing tools and frameworks (e.g., Selenium, Cypress, Playwright).
- Experience testing AI/ML-based applications, including validating model outputs, scoring accuracy, and consistency.
- Understanding of LLM concepts such as prompts, hallucinations, bias, and non-deterministic outputs, and how to design test strategies around them.
- Familiarity with AI-powered testing tools (e.g., Testim, Mabl, Applitools, GitHub Copilot) for smarter and faster testing.
- Experience with API testing (e.g., Postman, REST Assured), including testing third-party AI APIs.
- Coding experience in languages such as Java, PHP, Python, or JavaScript, along with HTML5 and XML.
- Solid knowledge of SQL and scripting.
- Experience writing clear, concise, and comprehensive test plans and test cases.
- Familiarity with web security technologies and QA tools and processes.
- Experience working in an Agile/Scrum development environment.
- Excellent analytical, problem-solving, time-management, and communication skills.
- Experience with performance and/or security testing is a plus.