About the role
Responsibilities:
- Lead, mentor, and direct-manage a team of 6-10 software and AI engineers, driving engineering quality, roadmap execution, and production reliability.
- Own end-to-end production shipping and maintenance of multi-step, tool-calling LLM agents that serve real production users.
- Establish technical standards and release processes for AI development, including CI/CD eval gates, shadow/canary rollouts, regression testing, and rollback strategies.
- Design, implement, and maintain rigorous evaluation frameworks and golden datasets (deterministic, LLM-as-judge, human-in-the-loop) to measure agent quality, task success rates, and precision/recall/F1 metrics.
- Exercise strong technical judgment across prompt design, model selection (balancing accuracy vs. cost vs. latency trade-offs), agent orchestration, and guardrails.
- Oversee the operational health, tracing, latency optimization (p95/p99), drift monitoring, and compute cost of production agentic systems.
- Partner closely with product, domain experts, and data leadership to define architectural strategy, technical direction, and business deliverables.
Requirements:
- 9+ years of overall software engineering experience, including at least 2+ years of hands-on people management, managing individual contributors (ICs).
- A strong backend engineering background, with prior hands-on experience owning and scaling production services before moving into a management role.
- At least 1+ year of experience leading teams that have built and shipped multi-step LLM agents using LLMs and tool-calling into real-world production environments.
- Strong hands-on understanding of AI evaluation.
- Good understanding of agentic AI frameworks and tracing/evaluation tools such as LangGraph, LangSmith, and Braintrust.
- Strong technical judgment to evaluate model choices, challenge agent or prompt designs, and troubleshoot production-level agent failures.
- Proven leadership experience in hiring, mentoring engineers, setting engineering standards, and driving technical excellence, while still maintaining strong technical depth.