About the role
Key Responsibilities: • Design and architect enterprise-scale Databricks Lakehouse solutions aligned with business, analytics, AI, and data modernization objectives. • Define scalable and secure data platform architectures leveraging Delta Lake, Unity Catalog, Medallion Architecture, and Databricks Workflows. • Architect cloud-native data solutions on Azure and AWS, integrating services such as ADLS, Azure Event Hubs, Azure Data Factory, Amazon S3, MSK/Kafka, IAM, and cloud-native security frameworks. • Design batch and real-time data processing solutions using Auto Loader, Delta Live Tables (DLT), Structured Streaming, Kafka, and Event Hubs. • Establish enterprise data governance, security, lineage, privacy, and compliance frameworks utilizing Unity Catalog, fine-grained access controls, and audit capabilities. • Lead the architecture and implementation of GenAI, Agentic AI, RAG, Mosaic AI, Vector Search, MLflow, and Databricks Genie solutions. • Define MLOps, AI governance, and model lifecycle management strategies covering explainability, observability, risk management, and regulatory compliance. • Drive cloud modernization and migration initiatives from legacy and on-premise platforms to Databricks-based lakehouse architectures. • Establish DataOps, CI/CD, Infrastructure as Code (Terraform), and platform automation standards for enterprise-scale deployments. • Optimize platform performance, scalability, resilience, and cost efficiency through architecture best practices, workload management, and FinOps principles. • Conduct architecture reviews, define enterprise standards, assess technical risks, and ensure alignment with organizational governance frameworks. • Provide technical leadership, stakeholder engagement, executive-level advisory, solution roadmap definition, and mentoring to engineering and architecture teams. • Key Skills / Technologies • Databricks: Delta Lake, Unity Catalog, DLT, Auto Loader, Databricks Workflows, Delta Sharing, Lakehouse Monitoring, Databricks Genie, Mosaic AI, MLflow, Vector Search • Cloud: Azure (ADLS, Event Hubs, ADF, Key Vault, Azure DevOps), AWS (S3, IAM, Glue, MSK, Lambda, CloudWatch) • AI/GenAI: RAG, AI Agents, LLMs, Vector Search, Prompt Engineering, Model Governance • DevOps: Terraform, CI/CD, DataOps, GitHub Actions, Azure DevOps