NextHire Try for free

Deloitte

Posted 6 Nov 2026

Senior Databricks Engineer

Apply with NextHire Auto Apply fills in the application for you.
Employment
Full time
Team
Software Engineering
Posted
6 Nov 2026

About the role

Job Summary:


We are seeking an experienced Databricks Developer with 6-9 years of expertise in designing, building, and supporting modern Data Lakehouse solutions. The ideal candidate will have strong hands-on experience in Databricks, PySpark, Delta Lake, cloud platforms, and large-scale data processing. The role involves building scalable data pipelines, optimizing distributed data workloads, and enabling advanced analytics and AI-driven initiatives.


Key Responsibilities:

  • Design, develop, and maintain scalable data engineering solutions using Databricks.
  • Build and optimize batch and real-time data processing pipelines.
  • Develop ETL/ELT frameworks using PySpark and Spark SQL.
  • Implement Delta Lake architecture and data governance standards.
  • Process large-scale structured, semi-structured, and streaming datasets.
  • Optimize Spark jobs and cluster performance for maximum efficiency.
  • Develop reusable frameworks, notebooks, workflows, and data products.
  • Collaborate with business teams, data scientists, analysts, and architects to deliver enterprise-grade solutions.
  • Implement monitoring, alerting, security, and compliance requirements.
  • Participate in code reviews, deployment activities, and production support.
  • Develop technical documentation and mentor junior engineers.


Required Technical Skills:


Databricks & Big Data:

  • Azure Databricks
  • Databricks Lakehouse Platform
  • Delta Lake
  • Spark SQL
  • PySpark
  • Databricks Workflows
  • Unity Catalog
  • Delta Live Tables (DLT)
  • Structured Streaming


Data Engineering:

  • Data Lake Architecture
  • Lakehouse Architecture
  • ETL/ELT Development
  • Data Modeling
  • Data Integration
  • Performance Tuning
  • Data Quality Frameworks


Programming Languages:

  • Python
  • PySpark
  • SQL
  • Scala
  • Java
  • Shell Scripting


Cloud Platforms:

  • Microsoft Azure
  • Azure Data Factory (ADF)
  • Azure Data Lake Storage (ADLS)
  • AWS
  • Amazon S3
  • AWS EMR
  • Google Cloud Platform (GCP)


Databases:

  • SQL Server
  • Oracle
  • PostgreSQL
  • MySQL
  • MongoDB
  • Cassandra


Streaming & Messaging:

  • Kafka
  • Event Hubs
  • Spark Streaming
  • Azure Stream Analytics


DevOps & Tools:

  • Git
  • Jenkins
  • Azure DevOps
  • CI/CD Pipelines
  • Terraform
  • Unix/Linux


Required Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or related field.
  • 4-9 years of IT experience with at least 4+ years of hands-on Databricks and Spark development experience.
  • Strong expertise in distributed data processing and Lakehouse architecture.
  • Experience building enterprise-scale data engineering solutions.
  • Strong knowledge of PySpark, Spark SQL, and performance optimization.
  • Experience with Agile development methodologies.
  • Excellent analytical, troubleshooting, and communication skills.


Preferred Qualifications:

  • Experience with cloud-native analytics platforms on Azure, AWS, or GCP.
  • Knowledge of Machine Learning workflows within Databricks.
  • Experience implementing Delta Lake and Unity Catalog.
  • Exposure to Data Governance and Data Security frameworks.
  • Relevant Databricks, Azure, or AWS certifications.


Key Competencies:

  • Data Engineering Excellence
  • Performance Optimization
  • Solution Design
  • Stakeholder Management
  • Collaboration and Leadership
  • Ownership and Accountability


Nice to Have:

  • Snowflake
  • Azure Data Factory (ADF)
  • Delta Live Tables (DLT)
  • Unity Catalog
  • MLflow
  • Airflow
  • Kafka
  • Data Mesh Architecture
  • Generative AI / AI Engineering exposure
  • dbt

Similar jobs

See every job

85% of jobs are filled through networking

The AI Outreach Agent writes to recruiters and hiring managers at the companies you pick, from your own Gmail.

Try AI Outreach