Machine Learning Engineer, Model Evaluation

XenonStack is a Data and AI Foundry for Agentic Systems, enabling enterprises to design, deploy, operate, and scale intelligent agents across digital and physical environments.

We build enterprise-grade platforms across the agentic stack:

• Akira AI — Reasoning and agent orchestration. Turn models into collaborative, policy-governed agents that learn and act together.
  
• ElixirData — Agentic analytics intelligence. Explainable, decision-centric analytics for measurable business outcomes.
  
• NexaStack — Agentic infrastructure automation. Secure, compliant AI deployment across cloud, edge, and on-prem.
  
• MetaSecure — Trust, compliance and defense. Continuous assurance with AI-BOMs, risk scoring, and agentic security.
  

Our mission is to accelerate the world’s transition to AI + Human Intelligence by making agentic systems reliable, responsible, and enterprise-ready.

We are seeking anMachine Learning Engineer, Model Evaluationto ensure that large language models (LLMs) and agentic AI systems meetenterprise-grade standards of accuracy, safety, and trustworthiness.

This role focuses onevaluating, benchmarking, and stress-testingLLMs in real-world workflows, building frameworks forreliability, robustness, and continuous improvement. If you thrive at the intersection ofAI research, applied testing, and responsible deployment, this is the role for you.

• Evaluation Frameworks
  

  • Design and implementLLM evaluation pipelinescovering accuracy, robustness, safety, and bias.
    

  • Develop automated systems forbenchmarking modelson enterprise-relevant tasks.
    

• Reliability Engineering
  

  • Conductstress tests, adversarial testing, and edge-case evaluations.
    

  • Build tools to measurelatency, consistency, and error recoveryin multi-turn interactions.
    

• Metrics & Monitoring
  

  • Define KPIs such asfactual accuracy, hallucination rate, toxicity, and compliance alignment.
    

  • Establish real-time monitoring fordrift, anomalies, and performance regressions.
    

• Collaboration & Alignment
  

  • Partner withML engineers, product managers, and domain expertsto align evaluation with business objectives.
    

  • Work with Responsible AI teams to implementethical, explainable, and compliant evaluation practices.
    

• Continuous Improvement
  

  • Feed insights from evaluation intofine-tuning, RLHF/RLAIF pipelines, and model selection.
    

  • Maintain acentral repository of test cases, benchmarks, and evaluation results.
    

• Research & Innovation
  

  • Stay current withstate-of-the-art LLM evaluation techniques, from academic benchmarks to applied enterprise metrics.
    

  • Exploreautomated evaluation using agentic test harnesses and synthetic data generation.
    

Must-Have

• 3–6 years inAI/ML, NLP, or applied model evaluation.
  

• Strong understanding ofLLM architectures, prompt engineering, and failure modes.
  

• Hands-on withevaluation frameworks(Eval harnesses, Ragas, OpenAI Evals, DeepEval).
  

• Proficiency inPythonand libraries likeLangChain, LangGraph, LlamaIndex, Hugging Face.
  

• Experience withvector databases, RAG pipelines, and knowledge graph integration.
  

• Familiarity withbias/fairness testing and Responsible AI frameworks.
  

Good-to-Have

• Experience withreinforcement learning (RLHF, RLAIF)and reward modeling.
  

• Exposure toagentic evaluation frameworks(multi-agent stress testing, synthetic user simulators).
  

• Knowledge ofcompliance and safety requirementsfor BFSI, GRC, or SOC use cases.
  

• Contributions toopen-source evaluation libraries or research papers.
  

1. 
   

   Agentic AI Product Company
   Ensure reliability in cutting-edge AI platforms that are redefining enterprise adoption.
   

   
   

2. 
   

   A Fast-Growing Category Leader
   Be part of one of the fastest-growingAI Foundries, powering Fortune 500 enterprises with trustworthy AI.
   

   
   

3. 
   

   Career Mobility & Growth
   Grow into roles such asAI Systems Architect, Responsible AI Engineer, or Reliability Engineering Lead.
   

   
   

4. 
   

   Global Exposure
   Work onenterprise-scale evaluation challengesacross BFSI, Healthcare, Telecom, and GRC.
   

   
   

5. 
   

   Create Real Impact
   Your evaluations will directly shapeproduction-grade AI agents used in mission-critical systems.
   

   
   

6. 
   

   Culture of Excellence
   Our values —Agency, Taste, Ownership, Mastery, Impatience, and Customer Obsession— empower you to innovate fearlessly.
   

   
   

7. 
   

   Responsible AI First
   Join a company that prioritizestrustworthy, explainable, and compliant AI.
   

   
   

At XenonStack, we believe inshaping the future of intelligent systems. We foster aculture of cultivationbuilt on bold, human-centric leadership principles, wheredeep work, simplicity, and adoptiondefine everything we do.

Our Cultural Values

• Agency– Be self-directed and proactive.
  

• Taste– Sweat the details and build with precision.
  

• Ownership– Take responsibility for outcomes.
  

• Mastery– Commit to continuous learning and growth.
  

• Impatience– Move fast and embrace progress.
  

• Customer Obsession– Always put the customer first.
  

Our Product Philosophy

• Obsessed with Adoption– Making AI accessible, reliable, and enterprise-ready.
  

• Obsessed with Simplicity– Turning complex evaluation challenges into seamless, automated frameworks.
  

Be part of our mission toaccelerate the world’s transition to AI + Human Intelligence— by making AI agents not just powerful, buttrustworthy and reliable.