About the role
We are seeking a highly motivated and detail-oriented Quality Assurance (QA) Engineer to join our team. You will play a key role in ensuring the reliability, accuracy, and user experience of our conversational AI systems. This role involves testing dialogue models, integrations with NLP/LLM-based backends, and end-to-end conversational workflows across multiple platforms (chat, voice, and integrations with enterprise systems).
The core responsibilities for the job include the following:
Test Planning and Strategy:
- Develop and execute comprehensive test plans for conversational AI features, including NLP pipelines, dialogue flows, integrations, and APIs.
- Design functional, regression, performance, and exploratory test cases specific to conversational interfaces.
Conversational Testing:
- Validate intent recognition, entity extraction, context handling, and multi-turn dialogue management.
- Test conversational flows for accuracy, completeness, and user experience.
- Perform scenario-based and edge-case testing to capture unexpected user inputs.
Automation and Tooling:
- Build automated test scripts for API endpoints, dialogue flows, and ML model outputs.
- Work with frameworks like Postman, Cypress, PyTest, or custom NLP testing tools.
- Contribute to CI/CD pipelines with automated regression tests.
- Collaborate with data scientists, ML engineers, and product managers to define acceptance criteria for AI-driven features.
- Provide feedback on conversational design and model performance.
- Collaborate with DevOps on load testing and monitoring production systems.
Quality Advocacy:
- Ensure conversational AI products meet high standards for accuracy, robustness, fairness, and usability.
- Help define and track KPIs (e. g., intent accuracy, response latency, conversation success rate).
Requirements:
- 3+ years of QA/testing experience, preferably with AI, ML, or NLP products.
- Strong understanding of API testing, automated frameworks, and scripting languages (Python, JavaScript, or similar).
- Experience testing chat agents, voice agents, or conversational interfaces is a plus.
- Familiarity with NLP concepts: intents, entities, context, LLMs, and dialogue management frameworks (e. g., Rasa, Dialogflow, and LLM APIs).
- Experience with CI/CD pipelines and version control (Git).
- Excellent problem-solving, analytical, and communication skills.
- Exposure to A/B testing and model evaluation.
- Experience with cloud platforms (AWS, GCP, Azure).
- Familiarity with accessibility and localization testing for multilingual agents.