About the role
We are looking for an experienced Data Engineer with 8+ years of hands-on experience in designing, developing, and maintaining scalable data engineering solutions.
The ideal candidate should have a strong Python and Data Engineering background, with hands-on expertise in Snowflake, ETL/ELT pipelines, Databricks/PySpark, Medallion Architecture, and data quality validation.
The role requires someone who can independently understand requirements, design data solutions, build production-grade pipelines, and take ownership of delivery end-to-end.
Key Responsibilities
- Design, develop, and maintain robust ETL/ELT data pipelines for large-scale data processing.
- Develop production-grade data engineering solutions using Python, SQL, Snowflake, and Databricks/PySpark.
- Work with Medallion Architecture (Bronze, Silver, Gold layers) for building scalable and reliable data platforms.
- Design and implement data transformation, cleansing, validation, and enrichment processes.
- Implement Data Completeness, Data Consistency, Data Deduplication, and Data Quality (DCDQ) checks across data pipelines.
- Ensure data accuracy, reliability, consistency, and availability across different stages of the data pipeline.
- Work with Snowflake for data storage, transformation, optimization, and performance tuning.
- Build and optimize data processing pipelines using Databricks and PySpark.
- Develop complex SQL queries and perform data modelling for analytical workloads.
- Integrate data from multiple sources and build scalable data pipelines.
- Troubleshoot pipeline failures, data-quality issues, and performance bottlenecks.
- Collaborate with product, engineering, and business stakeholders to understand requirements and translate them into effective data solutions.
- Independently drive projects from requirement gathering → solution design → development → testing → deployment → production support.
- Follow engineering best practices around code quality, testing, monitoring, documentation, and maintainability.
Required SkillsData Engineering
- 8+ years of strong hands-on experience in Data Engineering.
- Strong experience designing and developing production-grade ETL/ELT pipelines.
- Strong understanding of data pipelines, data warehousing, data modelling, and data transformation.
- Experience working with Medallion Architecture – Bronze, Silver, and Gold layers.
- Strong understanding of data quality, validation, reconciliation, and DCDQ checks.
- Strong hands-on Python programming experience.
- Advanced SQL skills.
- Hands-on experience with Apache Spark / PySpark.
- Ability to write clean, reusable, maintainable production code.
- Strong hands-on experience with Snowflake.
- Experience with Databricks and PySpark.
- Good understanding of modern data warehouse/lakehouse concepts.
- Experience with Delta Lake or similar technologies is a plus.
- Experience building and managing batch data pipelines.
- Experience with data pipeline orchestration tools such as Airflow, Databricks Workflows, or similar.
- Experience with streaming technologies such as Kafka is a plus.
- Strong understanding of data quality frameworks and validation mechanisms.
- Experience implementing checks for:
- Data completeness
- Data consistency
- Data duplication
- Data accuracy
- Data reconciliation
- Ability to identify and troubleshoot data-quality issues across different pipeline stages.
- Exposure to AWS, Azure, or GCP is good to have but not mandatory.
- Experience with CI/CD, Docker, Terraform, or Kubernetes is a plus.
- Experience with dbt is a plus.
What We Are Looking For
- Strong Python-first Data Engineering background.
- 8+ years of relevant experience with significant hands-on development.
- Strong Snowflake + ETL/ELT experience.
- Good hands-on experience with Databricks/PySpark.
- Practical understanding of Medallion Architecture.
- Strong focus on data quality and DCDQ checks.
- Ability to independently own data engineering projects from requirements through production.
- Strong problem-solving and analytical skills.
- Good communication and stakeholder-management skills.
- Ability to understand business requirements, ask the right questions, and convert them into scalable technical solutions.