Lead Data Analyst

As a Lead Databricks engineer, this role needs an experienced Data professional with 8+ years of expertise in Data Architecture, Data Modeling, and Cloud Analytics solutions, specializing in designing scalable enterprise data platforms using Databricks, Apache Spark, and cloud technologies. Strong background in building Lakehouse architectures, dimensional data models, ETL/ELT pipelines, and high-performance analytics solutions across enterprise environments. Skilled in data governance, performance optimization, cloud migration, and implementing modern data engineering best practices on Microsoft Azure and Amazon Web Services. Proven ability to lead cross-functional teams, architect end-to-end data solutions, and deliver scalable platforms supporting BI, reporting, and AI/ML initiatives.

• Design and document end-to-end architecture for data analytics platforms leveraging Databricks on Azure/AWS, ensuring alignment with business and technical requirements
• Design multi-layered data architecture (Bronze, Silver, Gold) aligned to best practices for ingestion, transformation and consumption
• Set up and enforce governance using Unity Catalog, establishing centralized access control, auditing, lineage, and data quality monitoring for all data assets
• Develop strategies and interfaces for seamless integration with existing client systems, enabling interoperability between Databricks and Azure/AWS native services, reducing data silos and aligning with cloud and security best practices
• Define and implement security models, policies, encryption standards, and access management using best practices in Azure/AWS and Databricks for regulatory compliance and risk mitigation
• Leverage Databricks Data Quality features and Unity Catalogs lineage functionality to maintain end-to-end traceability and ensure trusted, accurate data flows across the platform
• Lead migration from on-prem DW systems to cloud platforms like Databricks, Snowflake, or Microsoft Azure
• Architect and guide the creation of automated MLOps pipelines for the full machine learning lifecycle, from feature engineering and model training to scalable deployment and continuous monitoring.
• Define and implement a framework for model governance using MLflow, including experiment tracking, model versioning, and a centralized model registry to ensure reproducibility and accountability.
• Design robust solutions for deploying trained models for batch inference, leveraging services like Databricks Model Serving or Amazon SageMaker to meet performance and cost requirements.
• Develop strategies for integrating data and models with advanced AI/ML services like Amazon Bedrock, enabling the development of generative AI and large language model (LLM) applications

• 8+ years of enterprise software experience, with 
         at least 5+ years in data, data modelling, data warehouse and ETL’s
         3+ years hands on experience with Databricks
  • Strong experience with Databricks on Azure/AWS cloud, with proven experience in designing data Lakehouse architecture
  • Expertise with Unity Catalog for centralized governance, RBAC, auditing, access control and lineage
  • In-depth knowledge of ADLS Gen2, Key Vault, AWS Lake Formation, S3, Glue, Lambda and orchestration using MWAA
  • Deep expertise in PySpark, Spark, Data Quality frameworks and distributed data processing
  • Expertise in designing and implementing MLOps practices using tools like MLflow for experiment tracking, model registry, and scalable deployment.
  • Proven experience with core Azure/ AWS AI/ML services, including Amazon SageMaker for model building and deployment, and Amazon Bedrock for generative AI applications.
  • Ability to architect solutions that seamlessly integrate Databricks with native Azure/AWS AI/ML services

• Experience with other Azure services such as Azure Data Factory, Azure Synapse Analytics, and Azure Machine Learning.
• Knowledge of programming languages such as Python or Scala for data processing and automation.
• Understanding of data warehousing concepts and experience with ETL processes.
• Familiarity with DevOps practices and tools for CI/CD in data engineering.

• Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.
• Relevant certifications in Azure or data engineering are a plus.
• Strong analytical and problem-solving skills with a keen attention to detail.
• Ability to adapt to changing priorities and work in a fast-paced environment.
• Experience with GenAI and AI engineering platforms
• Insurance or Financial Services domain exposure
• Knowledge of:
  • Real-time analytics
  • MLOps
  • Data mesh
• Databricks certifications:
  • Databricks Certified Data Engineer
  • Databricks Solution Architect

Mphasis applies next-generation technology to help enterprises transform businesses globally. Customer centricity is foundational to Mphasis and is reflected in the Mphasis’ Front2Back™ Transformation approach. Front2Back™ uses the exponential power of cloud and cognitive to provide hyper-personalized (C=X2C2TM=1) digital experience to clients and their end customers. Mphasis’ Service Transformation approach helps ‘shrink the core’ through the application of digital technologies across legacy environments within an enterprise, enabling businesses to stay ahead in a changing world. Mphasis’ core reference architectures and tools, speed and innovation with domain expertise and specialization are key to building strong relationships with marquee clients.

Mphasis is an equal opportunity/affirmative action employer. We provide equal employment opportunities to applicants and existing associates and evaluate qualified candidates without regard to race, gender, national origin, ancestry, age, color, religious creed, marital status, genetic information, sexual orientation, gender identity, gender expression, sex (including pregnancy, breast feeding and related medical conditions), mental or physical disability, medical conditions military and veteran status or any other status or condition protected by applicable federal, state, or local laws, governmental regulations and executive orders. View the EEO in the law poster here, view the EEO in the law supplement here. To view the pay transparency nondiscrimination provision please click here and to view the E-Verify posting click here.
Mphasis is committed to providing reasonable accommodations to individuals with disabilities.