About the role
Key Responsibilities:
- Design and build LLM agents that plan, call tools and complete multi-step business tasks across voice, chat and back-office channels.
- Implement task and conversation state, memory and context management, retrieval (RAG) and model routing between fast and more capable models.
- Define tool schemas and permissions, validate every tool call, and add human approval points for high-impact actions.
- Build durable workflows with checkpoints, retries, idempotency keys, timeouts and compensation for failed actions.
- Use multi-agent designs, such as a router with specialist agents, only where they measurably beat a single agent.
- Integrate CRMs, ticketing systems and databases for actions such as complaint registration, classification, routing, status updates and notifications, with warm human handoff.
Voice Agents:
- Build streaming pipelines that connect telephony (SIP, Twilio, Exotel) and WebRTC audio to ASR, LLMs and TTS over WebSockets.
- Implement voice activity detection, turn detection, barge-in and cancellation so the agent stops talking the moment a caller speaks.
- Set and hold latency budgets per hop (ASR, LLM, TTS, network), with a target of under [1 s] from end of caller speech to first agent audio.
- Adapt agent behaviour for speech: short turns, confirmations before actions, and recovery from mishearing.
- Support multilingual and code-switched calls, including language detection and matching TTS voices.
Evaluation:
- Build and maintain golden test sets that define the user goal, persona, expected tool calls and pass criteria for each scenario.
- Run trajectory evals that check each step's tool choice, arguments and order, not just the final answer.
- Run simulated-caller evals for voice: an LLM plays the caller, TTS voices it with accents, noise and interruptions, and the agent is tested end to end over real audio.
- Score task success, tool-call correctness, policy compliance, hallucination, word error rate and latency, using rule checks and LLM judges calibrated against human ratings.
- Gate releases in CI on eval results, block regressions, and turn every confirmed production failure into a new test case.
Backend and Operations:
- Write async Python services (FastAPI, asyncio) with structured logging, tracing and metrics for every agent run and conversation.
- Load-test concurrent sessions, find bottlenecks and keep p95 latency and per-task model cost within budget.
- Protect sensitive data: authentication and authorisation, audit trails for agent actions, PII redaction and tenant isolation.
- Ship with Docker and CI using staged releases and rollback, and troubleshoot live agents in production.
- Work with product and delivery teams to turn client workflows into working agent integrations.
Technical Leadership:
- Own the architecture of agent and voice systems, document key decisions and explain trade-offs to engineers, product and clients.
- Break client requirements into technical designs and delivery milestones, and assign work across the team.
- Review code and designs, and set team standards for testing, evaluation, monitoring and release.
- Mentor AI engineers through pairing, design reviews and feedback, and help them grow into owning features end to end.
- Lead incident response for production issues, run blameless post-mortems and drive the fixes to completion.