About the role
Responsibilities:
- Design and develop scalable data pipelines on GCP.
- Build ETL/ELT solutions using Dataflow and Dataproc.
- Develop and optimise BigQuery datasets for reporting and analytics.
- Manage structured and unstructured data in GCS.
- Implement data quality, validation, and monitoring frameworks.
- Collaborate with business, analytics, and engineering teams to deliver data solutions.
- Troubleshoot production issues and optimise pipeline performance.
Requirements:
- Strong SQL expertise.
- Python and PySpark development.
- Apache Spark ecosystem.
- BigQuery, Dataflow, Dataproc, GCS (mandatory).
- Data warehousing and data modelling concepts.
- Experience with batch and real-time data processing.
- Understanding of cloud security and IAM.