About the role
Azure Data Engineer:
Key Skills:- Azure Databricks, Azure Data Factory, SQL, PySpark , Python
Roles & responsibilities
Operational:
- Design, develop, and support data solutions using Azure platform to enhance operational efficiency and data integrity.
- Utilize expertise in Python, PySpark, and SQL to analyze complex datasets and generate insightful reports that drive improvements in client operations.
- Implement configurations and develop technical solutions that automate processes and ensure seamless integration of data across various cloud platforms.
- Functional:
- Build ETL workflows and automate data pipeline tasks using Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and AWS Glue.
- Develop scripts and automation tools for data ingestion and transformation, leveraging PySpark and other libraries to streamline processes.
- Create and manage robust data processing applications, ensuring effective data exchange and integration between systems.
Mandatory technical & functional skills
- Python: Advanced skills in data processing using PySpark.
- SQL: Solid knowledge for database management and querying.
- Cloud Environments: Extensive experience in Azure services (Data Factory, Synapse, Databricks) and AWS (Glue, Lambda).
- Data Design: Expertise in designing and implementing cloud-based data solutions.
Preferred technical & functional skills
- CI/CD: Experience with Azure DevOps for automated deployment.
- Data Manipulation: Proficient use of PySpark for data transformation.
- Version Control: Familiar with Git for code management.
- Big Data Tools: Knowledge of Apache Airflow for ETL orchestration.