About the role
Responsibilities:
- Own the technical design and architecture of scalable data-processing and transformation solutions on GCP.
- Design and build production-grade batch, incremental and streaming data pipelines.
- Develop and optimise complex transformations using BigQuery, SQL and Dataform / dbt.
- Build reusable, metadata-driven and configuration-driven frameworks that reduce bespoke pipeline development.
- Design orchestration patterns using Cloud Composer / Apache Airflow, including dynamic workflows, dependencies, retries and recovery.
- Develop distributed and streaming processing solutions using Dataflow / Apache Beam and Dataproc / Spark.
- Define standards for data modelling, schema design, schema evolution and data contracts.
- Design and evolve modern analytical storage patterns, including Apache Iceberg.
- Establish engineering patterns for data quality, reconciliation, metadata, lineage and observability.
- Optimise large-scale workloads for performance, scalability, latency and GCP cost.
- Define reusable engineering standards for testing, versioning, CI/CD and production readiness.