About the role
The core responsibilities for the job include the following:
Data Platform (Core):
- Architect and scale a modern data lake supporting both batch and real-time (streaming) workloads.
- Design and manage reliable data ingestion pipelines.
- Data modeling and governance.
- Data quality and lineage.
- Build and maintain a scalable feature store for ML and AI systems.
- Develop a robust semantic layer that enables analytics, experimentation, and AI use cases.
ML and Data Science:
- Own the complete ML lifecycle from training and validation to deployment, monitoring, and retraining.
- Build and operationalize predictive and optimization models.
- Establish experimentation frameworks and model evaluation standards.
- Implement drift detection, monitoring, and performance tracking to ensure model reliability.
AI Readiness:
- Design and deploy RAG (Retrieval-Augmented Generation) pipelines.
- Build and manage embeddings infrastructure.
- Structure and maintain AI-ready datasets optimized for LLMs and agent-based systems.
- Implement feedback loops that enable continuous learning and system improvement.
What You'll Build:
- Central data lake (events, operational, external).
- Real-time pipelines (Kafka / PubSub / streaming).
- ML pipelines (forecasting, anomaly detection, optimization), feature stores, experiment tracking, data observability, decision intelligence dashboards, and AI-ready datasets.
Architecture First:
- Define the complete data architecture: source, ingestion, storage, transformation, and serving.
- Design and optimize offline vs. real-time processing strategies.
- Make informed cost vs. performance trade-offs.
ML in Production:
- Model deployment pipelines.
- Versioning.
- Monitoring.
- Retraining strategies.
Cross-Functional:
- Work with the product to define signals.
- Work with engineering to integrate pipelines.
- Working with AI teams enables agent intelligence.
Requirements:
- 6+ years of experience across data engineering, data science, or machine learning roles.
- Proven experience designing and building data lake or data warehouse architectures in production.
- Demonstrated ability to ship ML models to production.
- Strong understanding of data modeling, feature engineering, ML lifecycle, experimentation, and data governance.
- Hands-on programming experience with Python and SQL.
- Experience working with modern cloud data stacks on GCP or AWS.