GCP Data Engineer

Required Skills:

• Expertise in Google Data Engineering services – Big Query, Dataproc etc 

• Strong experience in programming languages such as Python, PySpark data manipulation and engineering tasks. 

• Expertise in SQL and NoSQL databases 

• In-depth knowledge of data warehousing solutions 

• Deep understanding on data warehousing concepts 

• Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache Airflow, Talend, or Informatica. 

Good to have skills:

• Understanding of DevOps Concepts in GCP. 

• Strong understanding of data modeling concepts and techniques to create efficient and scalable data models. 

• Experience with version control systems such as Git for code management and collaboration. 

• Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information.

SN 

Responsibility of / Expectations from the Role  

1 

Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility. 

2 

Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data. 

3 

Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency 

4 

Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle. 

5 

Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.