About the role
- Build up data pipelines for consumption by the data science team.
- Clear understanding and experience with Python and PySpark.
- Experience in writing Python programs and SQL queries.
- Experience in SQL Query tuning.
- Build and maintain data pipelines in Pyspark with SQL and Python.
- Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best practices in data integration.
- Split the planned deliverables into tasks and assign them to the team.
- Good oral, written and presentation skills.
Requirements
- Degree in Computer Science, IT, or a similar field; a master's is a plus. Hands-on experience with Python and Pyspark
- 3-5 years of hands-on experience in Python, PySpark, and SQL.
- Great numerical and analytical skills.
- Proven expertise in building and optimizing data pipelines using Spark and Hadoop ecosystem tools.
- Experience working on cloud platforms like Azure or AWS.
- Should be able to collaborate and coordinate in a remote environment. Be a problem solver and be proactive to solve the challenges that come his way.