About the role
Responsibilities:
- Design, develop, and manage end-to-end data pipelines using Azure Data Factory (ADF) for efficient ETL/ELT processes.
- Develop, deploy, and monitor scalable data solutions using Azure Synapse Analytics/Databricks/Microsoft Fabric to support large-scale data processing and analytics.
- Write optimised SQL Server scripts, stored procedures, and views to support data transformation and integration requirements.
- Build and maintain insightful dashboards and reports using Power BI, with a focus on usability, performance, and clear data communication.
- Develop automation scripts using PowerShell for operational and monitoring tasks across Azure services.
- Create automated workflows using Power Automate to streamline manual tasks and notifications.
- Leverage Python / PySpark for data manipulation, transformation, and orchestration within Synapse or Databricks environments.
- Collaborate with business stakeholders, data analysts, and application developers to gather requirements and deliver high-quality solutions.
- Perform root cause analysis and data validation, and ensure data integrity across systems and processes.
- Participate in code reviews, design discussions, and documentation efforts to ensure best practices and knowledge sharing.
- Ensure security, compliance, and governance in all data engineering activities.