About the role
As a Senior Data and Analytics Engineer, you will build, support, and optimise modern data solutions and analytical products across the Azure Data Platform. You'll design and manage robust data pipelines (Azure Data Factory, Databricks, Data Lake, Synapse). You will collaborate closely with business stakeholders, data engineers, and data scientists to deliver reliable, secure, and scalable data products that drive decision-making, particularly within Supply Chain and CPG contexts.
Responsibilities:
- Design, develop, customise, and manage data integration tools, data lakes, warehouses, and analytical systems on Azure.
- Build, scale, and harden data pipelines (internal & external sources) using Azure Data Factory, Azure Databricks (PySpark/Spark SQL), Azure Data Lake, Azure Synapse, and Azure SQL.
- Own automation, observability, and monitoring frameworks, capturing operational KPIs and data quality metrics; set up alerting and runbooks.
- Deploy pipelines to production, manage compute resources, configure data attributes, set up monitoring tools, and ensure resilient operations.
- Monitor performance & stability, triage incidents, and adapt pipelines as data/models/requirements evolve.
- Implement best practices for systems integration, security (RBAC, Key Vault, RLS/OLS), performance, cost optimisation, and data governance.
- Apply DevOps/DataOps/Agile methodologies; manage version control (Git/Bitbucket) and CI/CD pipelines (Azure DevOps/Jenkins) for data and BI assets.
- Partner with business, IT, and SMEs to translate requirements into scalable data products; lead solutioning and POC discussions.
Requirements:
- Preferred: Experience in CPG Supply Chain (planning, sourcing, manufacturing, logistics, customer service) and related KPIs.
- Bachelor's degree in Engineering, Computer Science, Data Analytics, IT, or related field.
- 4-6 years of relevant experience across Azure Data Engineering and Power BI.
- Hands-on expertise with: Azure Databricks (PySpark/Spark SQL)\Microsoft Fabric, Azure Data Factory, Azure Data Lake, Azure Synapse, Azure SQL. SQL/T-SQL (complex queries, stored procedures).
- Experience with Git/Bitbucket, Azure DevOps/Jenkins for CI/CD of data and BI assets.
- Strong scripting, data modelling, and programming fundamentals.
- Solid understanding of cloud architecture principles (security, scalability, reliability, cost).
- Excellent communication, stakeholder management, and the ability to mentor others.