About the role
Responsibilities:
- Backend Integration of LLM Architectures: Lead the development and maintenance of the backend infrastructure that powers state-of-the-art Large Language Model (LLM) architectures, including RAG, ReAct, and Agent-based systems. Architect data pipelines for processing extensive datasets and devise AI-powered endpoints crucial for our SaaS application and VSCode Extension.
- Graph-Based Data Structure Design: Design and develop large-scale graph-based data structures that effectively model complex data stores. Create intricate, efficient structures that can be fed to our AI applications as context, ensuring both scalability and performance.
- Intelligent SQL Ecosystem: Lead the design and development of a comprehensive SQL intelligence system encompassing query optimization, dynamic pipeline generation, and data lineage tracking. Leverage expertise in SQL query profiling, AST analysis, and parsing to create a sophisticated engine focused on query performance improvements and implementing granular column-level lineage.
Impact:
- Innovation at the Forefront: Push the boundaries of software engineering by combining traditional techniques with cutting-edge AI technologies.
- High Visibility and Impact: Directly affect the productivity and capabilities of global data teams, as your contributions will be crucial to the daily operations of thousands of users spread across 100s of countries.
- Open Source Contribution: As part of our commitment to the developer community, you will contribute to our open-source initiatives, gaining recognition in the tech community.
- Career Growth: This role is a launchpad into the rapidly advancing field of AI, offering exposure to state-of-the-art technologies and generative AI applications.
Requirements:
- 10+ years of experience building Python-based web application backends and the infrastructure on which they run.
- Extensive experience in building scalable back-end systems, APIs, and microservices.
- Deep understanding of cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes).
Preferred:
- Experience with FastAPI for building high-performance APIs.
- Familiarity with AI and machine learning concepts, particularly in the context of Large Language Models (LLMs).
- Strong proficiency in SQL, including query profiling, optimization, and performance tuning.
- Experience with SQL Abstract Syntax Tree (AST) analysis and working with SQL parsers (e. g., sqlglot).