About the role
Data Pipeline Development: Design, build, and optimize robust end-to-end data pipelines for data ingestion, transformation, and storage using GCP services and modern ETL/ELT frameworks. Data Warehousing & Modeling: Develop and optimize datasets, tables, views, and complex queries within cloud data warehouses (specifically Google BigQuery, Snowflake, or Databricks). Performance & Scalability Tuning: Monitor, troubleshoot, and tune data pipelines and queries to ensure optimal performance, low latency, and cost-efficiency. Infrastructure & Orchestration: Manage data workflows, orchestration, and infrastructure components using tools like Apache Airflow / Cloud Composer, Terraform, Docker, and Kubernetes. Data Quality & Governance: Implement data validation checks, monitoring solutions, alerting systems, and security protocols to ensure data integrity and compliance. Collaboration: Partner with data scientists, analysts, and DevOps teams to integrate data systems securely and reliably into production environments. Core Cloud Platform: Strong hands-on experience designing and operating solutions on Google Cloud Platform (GCP) (BigQuery, Dataflow, Cloud Storage, Dataproc, Cloud Composer). Data Warehouses & Databases: Expertise in Cloud Data Warehouses (Google BigQuery, Snowflake, AWS Redshift) and familiarity with NoSQL/relational databases. ETL & Processing Frameworks: Experience with ETL tools and data processing frameworks such as Apache Spark, PySpark, dbt, Fivetran, or Informatica. Programming & Querying: Advanced proficiency in SQL and programming languages like Python or Java for data manipulation. DevOps & Containers: Familiarity with containerization and orchestration tools (Docker, Kubernetes/GKE) and Infrastructure-as-Code (Terraform). Experience: 6+ years of relevant total experience in software development, database development, or big data engineering roles.