About the role
Position: Senior Data EngineerExperience: 5+ yearsWork Mode: Hybrid - PuneLocation: Preferred from Pune. (Relocation)Shift: Time zone overlap time with US 8:30 pm to 11:30 pm IST
Candidates with a strong background in streaming technologies such as Kafka, AWS MSK, AWS Kinesis, AWS Data Firehose, and Snowpipe streaming. Essential skills
- Ability to work independently
- CloudOps expertise for setting up infrastructure using IaC tools like Terraform and Pulumi
- Strong experience in building ETL pipelines.
- Understanding delta loading and CDC in Snowflake is also crucial
Responsibilities:
- Design, develop and maintain a data platform that is accurate, secure, available, and fast.
- Engineer efficient, adaptable, and scalable data pipelines to process data.
- Integrate and maintain a variety of data sources: different databases, APIs, SAASs, files, logs, events, etc.
- Create standardized datasets to service a wide variety of use cases.
- Develop subject-matter expertise in tables, systems, and processes.
- Partner with product and engineering to ensure product changes integrate well with the data platform.
- Partner with diverse stakeholder teams, understand their challenges and empower them with data solutions to meet their goals.
- Perform data quality on data sources and automate and maintain a quality control capability.
- At least 5+ years of experience as a Data Engineer
- You have hands-on and deep experience with Star/Snowflake schema design, data modeling, data pipelining and MLOps.- Experience in Data Warehouse technologies (e.g. Snowflake, AWS Redshift, etc)
- Experience in AWS data pipelines (Lambda, AWS glue, Step functions, etc)
- Fintech or Financial services industry experience
- You are proficient in SQL and at least one major programming language like Python, Java
- Experience with Data Analysis Tools such as Looker or Tableau
- Experience with Pandas, Numpy, Scikit-learn, and Jupyter notebooks preferred, familiarity with Git, GitHub, and JIRA.
- You have an eye for detail. You catch & resolve data quality issues before others do
- You are always looking for opportunities to simplify, automate tasks, and build reusable components. You use good judgment in applying new technologies to solve business problems