About the role
The role is responsible for leading the design and development of enterprise-grade GenAI capabilities for banking applications, with a strong focus on LLM inference, RAG, intelligent agents, and scalable backend systems. The role will drive the architecture and implementation of highly available, secure, and low-latency distributed systems that enable GenAI use cases across banking functions while optimizing model serving for performance, scalability, reliability, and cost. The role will work closely with product, ML/AI, data, and platform teams to take GenAI solutions from concept to production, ensuring they meet the security, compliance, reliability, and operational requirements of a banking environment.
Responsibilities:
- Design and build scalable backend and inference systems for production-grade GenAI applications, including LLM serving, APIs, and distributed services.
- Build and optimize RAG pipelines, including document ingestion, chunking, embeddings, vector search, retrieval, reranking, and context orchestration.
- Develop and productionize GenAI capabilities such as LLM inference, prompt engineering, agents, tool calling, fine-tuning, and model evaluation.
- Optimize inference performance across latency, throughput, GPU utilization, scalability, availability, and cost through techniques such as batching, caching, quantization, and efficient model serving.
- Build robust data and AI infrastructure covering data pipelines, model serving, observability, evaluation, monitoring, and continuous improvement of GenAI systems.
- Collaborate with ML, Product, Platform, and Infrastructure teams to take GenAI solutions from prototype to reliable, scalable production systems.
Secondary Responsibilities:
- Evaluate emerging LLM, GenAI, inference, and agentic AI technologies and identify opportunities for adoption.
- Contribute to POCs, technical prototypes, and architecture initiatives to validate new GenAI capabilities and approaches.
- Establish best practices for GenAI evaluation, prompt/version management, model quality, safety, and responsible AI.
- Improve engineering productivity through automation, reusable frameworks, tooling, and developer platforms for GenAI teams.
- Support production operations, incident resolution, capacity planning, and performance troubleshooting for GenAI services.
- Contribute to technical documentation, architecture standards, knowledge sharing, and engineering best practices across the organization.
Managerial and Leadership Responsibilities:
- Lead and mentor a team of backend and inference engineers building scalable, reliable, and high-performance GenAI systems.
- Own technical strategy and architecture for GenAI inference, model serving, backend services, APIs, and distributed systems.
- Drive end-to-end execution of engineering initiatives, including planning, prioritization, delivery, and cross-functional collaboration.
- Drive optimization of inference systems for latency, throughput, scalability, GPU utilization, reliability, and cost efficiency.
- Build a strong engineering culture through technical mentorship, performance management, hiring, and development of high-performing teams.
- Partner with product, ML/AI, and infrastructure teams to productionize GenAI capabilities and ensure operational excellence.