About the role
Key Responsibilities :
- Design, develop, and maintain ETL/ELT pipelines using Airflow and transformation tools like dbt (or similar)
- Manage data workflows and orchestration on Google Cloud Composer
- Build and maintain data storage and processing workflows, preferably on Google Cloud Platform (GCP) using BigQuery and GCS (or equivalent tools in other cloud environments)
- Automate infrastructure and deployments using Terraform
- Collaborate with data analysts, scientists, and engineers to deliver clean, tested, and reliable datasets
- Monitor pipeline performance, handle failures, and ensure data quality and observability
- Follow best practices for coding, testing, documentation, and version control
Required Skills and Experience
- Strong proficiency in Python for data processing and scripting
- Experience with Airflow (or similar orchestration frameworks)
- Familiarity with data transformation tools such as dbt, or comparable tools like Spark SQL, Dataform, etc.
- Experience with cloud data platforms, ideally GCP (BigQuery, GCS), or equivalent platforms (e.g., AWS S3/Redshift, Azure Data Lake/Synapse)
- Understanding of Terraform and infrastructure-as-code practices
- Solid foundation in data modeling, SQL, and data warehousing principles
- Comfortable working in collaborative, agile environments with Git-based workflows