About the role
Responsibilities:
- Take models and methods from research and turn them into production-ready services.
- Build evaluation frameworks to determine whether a change genuinely improves performance.
- Optimize retrieval, prompting, and memory for accuracy and cost efficiency.
- Instrument production systems to ensure that regressions are identified before they impact customers.
- Work across the entire development cycle, collaborating with research teams and providing feedback based on issues and observations from production.
Requirements:
- A working understanding of modern machine learning and LLMs, including how they work, where they fail, and why.
- Strong software engineering fundamentals, with the ability to write maintainable code that other engineers can run, understand, and modify.
- 2+ years of experience shipping real-world software or production systems, whether ML-backed or otherwise.
- Strong proficiency in Python and familiarity with PyTorch or an equivalent ML framework.
- Strong engineering judgment and the ability to distinguish genuine improvements from changes that only make performance metrics look better.
Nice to Have:
- Hands-on experience with retrieval, embeddings, or vector search in production environments.
- Experience with evaluation frameworks, LLM-as-a-Judge setups, and an understanding of their limitations.
- Exposure to fine-tuning techniques, including LoRA or adapters.
- Experience owning both a production service and a machine learning model.