About the role
• Design and build LLM-powered agents that plan tasks, call tools, and propose or apply actions against our services and data, under confidence thresholds and human-in-the-loop review
• Define the tools and APIs agents can use, and connect agents to internal systems (e.g., via Model Context Protocol)
• Build retrieval-augmented (RAG) pipelines: chunking, embeddings, vector search, and grounding responses in our data
• Design agent workflows with clear guardrails, confidence gating, human-in-the-loop exception handling, and fallback behavior
• Write prompts and evaluation harnesses; run evaluation in CI and measure agent quality, cost, and latency, then iterate
• Build and maintain Python services and shared libraries, and Kafka producers/consumers for moving data between services
• Serve agents behind APIs and monitor them with LLM/agent observability (quality, cost, latency, failures)
• Write design docs and collaborate with engineers and data scientists on architecture decisions
• Use AI coding agents in your own day-to-day workflow while owning correctness and quality Basic Qualifications:
- These are the core skills we screen for. A strong candidate has all three.
- Python 3 — solid experience writing production code, with strong fundamentals and readable, tested code, using a web framework such as FastAPI, Flask, or Django
- PySpark & SQL — hands-on experience writing and tuning Spark jobs on EMR (we run EMR on EKS), reading/writing Delta Lake tables, and strong SQL for large-scale data
- Agentic AI — hands-on experience building LLM or agent-based features: tool/function calling, multi-step agent workflows, RAG, and grounding output in real data.
- Experience writing production Python 3, including designing and optimizing large-scale data processing solutions
- Experience building services or APIs with a Python web framework such as FastAPI, Flask, or Django
- Hands-on experience with Spark / PySpark, ideally on EMR (EMR on EKS a plus)
- Strong SQL and familiarity with big data technologies
- Hands-on experience building LLM or agent-based features (tool calling, function calling, or multi-step agent workflows)
- Experience with an agent or LLM framework or platform such as Amazon Bedrock, LangChain, LangGraph, LlamaIndex, or the OpenAI/Anthropic SDKs
- Experience with RAG: embeddings, vector search, and grounding model output in real data
- Practical prompt engineering and experience evaluating LLM/agent output for quality and reliability
- Experience handling sensitive data and PII responsibly, including data-governance and privacy considerations when sending data to external LLMs
- Experience with at least one database (MongoDB/DocumentDB, PostgreSQL, or similar)
- Experience with AWS services such as S3 and EMR
- Experience writing tests with pytest (or similar) and working with Git
- Comfortable with Docker and Linux
- Practical experience using AI coding assistants or agents (e.g., Kiro, Cursor, GitHub Copilot, Claude Code) in real projects
- Strong communication: able to write design docs and collaborate across teams
- Experience with Model Context Protocol (MCP) or wiring agents to external tools and data sources
- Experience keeping LLM systems reliable: guardrails, evaluation pipelines, cost/latency tuning, caching
- LLM/agent observability and lifecycle experience: prompt/version management, monitoring, and quality regression detection
- Solid ML fundamentals and quality metrics (precision, recall, F1); familiarity with scikit-learn, pandas, NumPy
- Delta Lake, AWS Glue, and Athena
- Kafka streaming pipelines
- Kubernetes / EKS, Helm, and GitOps deployment
- Experience in a polyglot microservice environment (Node.js / Python / Java)
- Awareness of bias and explainability in AI systems