About the role
• Looking for candidates with an overall experience of 5 to 7 years
• Should have hands-on experience in PySpark programming coupled with extensive experience in Data Bricks environment.
• Should have minimum 3+ years of experience in Data Engineering, building ETL/ELT Data Pipelines.
• Should have experience in Relational/No SQL Database Management technologies.
• Should have experience in Azure Cloud technologies (Blob Storage, DataBricks, Delta Tables)
• Should have hands-on experience in Data validations, Quality checks.
• Should have strong working experience in Containerization using Kubernetes. (AWS EKS)
• Should have strong experience in data ingestion, transformation, cleansing/manipulation and onboarding the identified data sources with data pipelines.
• Should have hands-on experience in Data Migration (On-Prem to Cloud or Cloud to Cloud)
• Should be an experienced user of Jira, Git/GitHub repository.
• Should have experience of working in an Agile environment.
Must have skills -
• Hands-on work experience in Python programming coupled with extensive experience in is using Pandas, NumPy, SciPy and other pythonic data science libraries.
• Hands-on work experience in Azure Cloud technologies (Blob Storage, DataBricks, Delta Tables)
• Hands-on work experience in in Relational and No SQL Database Management technologies.
• Hands-on experience in data ingestion, transformation, cleansing/manipulation and onboarding the identified data sources with data pipelines.
• Hands-on work experience in Data Migration (On-Prem to Cloud or Cloud to Cloud)
Nice to have-
• Hands-On experience in MongoDB, Dynamo DB.
• Hands-On experience in PostgreSQL
• Hands-On experience in Data Manipulation & Harmonization using SQL
• Knowledge of Data Modelling, Data Governance, Data Cataloguing
• Knowledge on Azure event hub and kafka
Note: Candidate should be willing to go through coding test as part of job screening.
Weightages: