NextHire Try for free

YAL

Posted 17 Nov 2026

Senior AI Engineer

Apply with NextHire Auto Apply fills in the application for you.
Employment
Full time
Team
Software Engineering
Posted
17 Nov 2026

About the role

Key Responsibilities:

  • Design and build LLM agents that plan, call tools and complete multi-step business tasks across voice, chat and back-office channels.
  • Implement task and conversation state, memory and context management, retrieval (RAG) and model routing between fast and more capable models.
  • Define tool schemas and permissions, validate every tool call, and add human approval points for high-impact actions.
  • Build durable workflows with checkpoints, retries, idempotency keys, timeouts and compensation for failed actions.
  • Use multi-agent designs, such as a router with specialist agents, only where they measurably beat a single agent.
  • Integrate CRMs, ticketing systems and databases for actions such as complaint registration, classification, routing, status updates and notifications, with warm human handoff.


Voice Agents:

  • Build streaming pipelines that connect telephony (SIP, Twilio, Exotel) and WebRTC audio to ASR, LLMs and TTS over WebSockets.
  • Implement voice activity detection, turn detection, barge-in and cancellation so the agent stops talking the moment a caller speaks.
  • Set and hold latency budgets per hop (ASR, LLM, TTS, network), with a target of under [1 s] from end of caller speech to first agent audio.
  • Adapt agent behaviour for speech: short turns, confirmations before actions, and recovery from mishearing.
  • Support multilingual and code-switched calls, including language detection and matching TTS voices.


Evaluation:

  • Build and maintain golden test sets that define the user goal, persona, expected tool calls and pass criteria for each scenario.
  • Run trajectory evals that check each step's tool choice, arguments and order, not just the final answer.
  • Run simulated-caller evals for voice: an LLM plays the caller, TTS voices it with accents, noise and interruptions, and the agent is tested end to end over real audio.
  • Score task success, tool-call correctness, policy compliance, hallucination, word error rate and latency, using rule checks and LLM judges calibrated against human ratings.
  • Gate releases in CI on eval results, block regressions, and turn every confirmed production failure into a new test case.


Backend and Operations:

  • Write async Python services (FastAPI, asyncio) with structured logging, tracing and metrics for every agent run and conversation.
  • Load-test concurrent sessions, find bottlenecks and keep p95 latency and per-task model cost within budget.
  • Protect sensitive data: authentication and authorisation, audit trails for agent actions, PII redaction and tenant isolation.
  • Ship with Docker and CI using staged releases and rollback, and troubleshoot live agents in production.
  • Work with product and delivery teams to turn client workflows into working agent integrations.


Technical Leadership:

  • Own the architecture of agent and voice systems, document key decisions and explain trade-offs to engineers, product and clients.
  • Break client requirements into technical designs and delivery milestones, and assign work across the team.
  • Review code and designs, and set team standards for testing, evaluation, monitoring and release.
  • Mentor AI engineers through pairing, design reviews and feedback, and help them grow into owning features end to end.
  • Lead incident response for production issues, run blameless post-mortems and drive the fixes to completion.

Similar jobs

See every job

85% of jobs are filled through networking

The AI Outreach Agent writes to recruiters and hiring managers at the companies you pick, from your own Gmail.

Try AI Outreach