About the role
You will design, train, fine-tune, and deploy production-grade speech and voice AI models. This is not an API-integration-only role; you will work deeply with TTS/STT models, own the ML lifecycle, and improve latency, accuracy, and robustness in production.
The candidate will have responsibilities across the following functions:
Model Development and Fine-Tuning:
- Train and fine-tune STT/TTS models (e. g., Whisper, Conformer, VITS, HiFi-GAN, or similar)
- Build and optimise neural vocoders and acoustic models for natural, low-latency speech synthesis.
- Experiment with adapter-based approaches (LoRA, fine-tuning recipes) and distributed training on multi-GPU setups.
ML Pipelines and Evaluation:
- Design scalable ML pipelines for data preprocessing, feature engineering, training, evaluation, and deployment.
- Create evaluation frameworks to measure WER, entity F1 latency (P95/P99), MOS, PESQ, and conversational quality.
- Automate logging and benchmarking (e. g., MLflow) and maintain comparisons against commercial baselines.
Real-Time Voice Systems:
- Architect streaming speech pipelines targeting sub-300ms end-to-end latency for live assistant interactions.
- Integrate with telephony/real-time communication stacks and optimise for interruption handling, turn detection, and VAD.
Collaboration and Ownership:
- Work closely with backend, platform, and product teams to ship AI features into production.
- Take ownership of model performance, data quality, and continuous experimentation.
Requirements:
- Strong Python; experience with PyTorch/TensorFlow and deep learning for speech/audio.
- Hands-on experience training/fine-tuning STT/TTS models (Whisper, ESPnet, NeMo, Kaldi/K2 SpeechBrain, etc. ).
- Experience building ML pipelines, evaluation frameworks, and deploying models on the cloud (GCP/AWS).
- Understanding of real-time constraints, latency optimisation, and production ML systems.
Nice-to-Have:
- Experience with Indian language datasets and multilingual speech models.
- Familiarity with LiveKit Agents, Pipecat, or similar real-time voice frameworks.
- Background in fintech, BFSI, or regulated domains.