About the role
Responsibilities:
- Own the AI agent problem decomposition: Deeply understand LLM-based agents in healthcare. Deconstruct complex clinical research into well-defined agent tasks. Define reliability, evaluation metrics, run evals, and iterate on results end-to-end.
- Go deep on oncology data and problem formulation: Gain a deep understanding of cancer patient data (pathology, treatment, genomics, clinical notes). Collaborate closely with the NLP team to define precise, clinically data-grounded problem statements.
- Build prototypes and evaluation tools: Prototype and build POCs using tools like Cursor/Claude Code. Design internal apps to evaluate AI model outputs. Coordinate with data annotation for quality, and work with the AI team for model improvements.
- Bridge AI capabilities to product workflows: Collaborate with product managers to integrate AI into user-facing workflows. Leverage a deep understanding of AI agent capabilities to reimagine workflows and design ideal, genuinely AI-native user experiences, moving beyond merely 'bolted-on' AI.
- Own the feedback loop between annotation, evaluation, and improvement: Oversee the full AI lifecycle: define annotation, ensure output quality, evaluate, identify failure modes, and drive mitigation with AI engineering. You are accountable for model reliability in production, not just demos.
Requirements:
- 4+ years in product management with at least 1.5 years working directly on AI/ML products.
- Hands-on experience with AI engineering concepts: RAG pipelines, agents, MCPs, LLM APIs, and LLM evaluation frameworks.
- Ability to vibe code, comfortable using Cursor, Copilot, or similar tools to build quick prototypes, evaluation apps, and POCs independently.
- Experience formulating ML/NLP problem statements from messy, real-world data
- Strong understanding of how LLM-based agents work, how to decompose problems into agent architectures, and how to evaluate their reliability.
- Excellent communication skills with the ability to operate across AI engineering, product, design, and data annotation teams.
- Self-driven, with a bias toward action. You don't wait for permission to prototype, investigate, or push things forward.