About the role
We are seeking a highly skilled Azure Data Engineer to design, develop, and optimize scalable data solutions on the Azure cloud platform. The ideal candidate should have strong hands-on experience with Azure data services, Databricks, PySpark, and modern data engineering practices. The candidate should be confident in explaining past projects and demonstrating ownership of end-to-end data pipelines.
Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure data services.
- Build and optimize data ingestion and transformation workflows using Azure Data Factory (ADF) and Azure Databricks.
- Develop data processing solutions using PySpark and Delta Lake architecture.
- Implement and manage data storage solutions using Azure Data Lake Storage (ADLS).
- Create and manage data pipelines using Delta Live Tables (DLT).
- Implement data governance, security, and access controls using Unity Catalog.
- Write complex SQL queries and stored procedures and optimize database performance.
- Collaborate with business stakeholders, architects, and development teams to understand data requirements.
- Monitor, troubleshoot, and improve data platform performance and reliability.
- Participate in code reviews, solution design discussions, and best-practice implementations.
Requirements:
- Strong experience with Azure Data Factory (ADF).
- Hands-on experience with Azure Data Lake Storage (ADLS).
- Hands-on experience with Azure Databricks.
- Strong proficiency in PySpark.
- Experience with Delta Live Tables (DLT).
- Experience with Unity Catalog.
- Strong SQL skills.
- Excellent communication skills with the ability to explain project work and technical implementations confidently.
- Strong understanding of data engineering concepts, data modeling, and ETL frameworks.
- Experience working with large-scale structured and unstructured datasets.
- Hands-on experience in Spark performance tuning and optimization.
- Knowledge of data governance, data quality, and security best practices.
- Strong analytical, problem-solving, and stakeholder-management skills.
- Ability to work independently and in a collaborative environment.
- Good To Have: Azure Functions, Azure DevOps / GitHub Actions (CI/CD), Python, Scala, Azure Synapse Analytics, Data Warehousing Concepts, Snowflake, AWS, GCP.