About the role
We are seeking a hands-on Data Engineer with 5+ years of experience building scalable data pipelines, data warehouses, and cloud-based analytics platforms. The ideal candidate has strong SQL and Python skills, practical experience with ETL/ELT development, Apache Airflow orchestration, cloud data platforms, and enterprise data modelling.
Responsibilities:
- Build and maintain scalable ETL/ELT pipelines.
- Develop and optimise SQL transformations and warehouse queries.
- Create Airflow DAGs for orchestration and scheduling.
- Design source-to-target mappings and curated data layers.
- Support reporting, dashboards, and analytics use cases.
- Work with structured and semi-structured datasets.
- Monitor pipeline health, failures, and data quality.
- Collaborate with BI, analytics, and AI/ML teams.
- Contribute to cloud-based data platform development.
- Maintain technical documentation and data lineage.
The core requirements for the job include the following:
Core Skills and Experience:
- SQL: Advanced SQL development including joins, CTEs, window functions, optimisation, stored procedures, and data transformations.
- Python: Strong Python programming for ETL pipelines, automation, APIs, and data processing.
- ETL/ELT: Experience building robust ingestion and transformation pipelines from APIs, databases, flat files, and cloud sources.
- Apache Airflow: Hands-on experience designing DAGs, scheduling workflows, retries, monitoring, and dependency management.
- Cloud Platforms: Practical experience with AWS, Azure, or GCP data services.
- Data Modelling: Understanding of dimensional modelling, star schemas, fact/dimension tables, normalisation, and warehouse design.
Preferred Technical Stack:
- Languages: Python, SQL.
- Orchestration: Apache Airflow.
- Cloud Platforms: At least one of: Amazon Web Services, Microsoft Azure, Google Cloud Platform.
- Databases / Warehouses: Snowflake, BigQuery, Redshift, SQL Server, PostgreSQL.
- Data Processing: Pandas, PySpark (nice to have).
- Version Control: Git.
Good Indicators of a Strong Candidate:
- Has built production-grade pipelines end-to-end.
- Understands performance optimisation and debugging.
- Can independently write complex SQL.
- Comfortable working with cloud-native data services.
- Understands how data supports reporting and AI workflows.
- Has worked with enterprise-scale datasets.
- Demonstrates strong problem-solving and ownership.