About the role
We are looking for a skilled and motivated Data Engineer to join our team. The ideal candidate will have strong technical expertise in SQL, Python, and PySpark, along with experience working with large, complex datasets. You will play a critical role in building and maintaining data pipelines, ensuring high data quality, and enabling strategic insights for stakeholders.
Responsibilities:
- Develop and maintain ETL pipelines using SQL, Python, and PySpark.
- Work with high-volume, complex data sets across various sources.
- Ensure data accuracy, consistency, and performance in processing workflows.
- Collaborate directly with stakeholders to understand business needs and deliver actionable insights.
- Write efficient and optimised code for large-scale data processing tasks.
- Schedule and monitor data jobs using job schedulers or orchestration tools.
- Contribute to designing scalable data solutions within the Hadoop ecosystem.
Requirements:
- 3-4 years of hands-on experience with SQL, Python, and PySpark.
- Strong understanding of data structures, data processing, and pipeline development.
- Familiarity with the Hadoop ecosystem (basic level).
- Excellent problem-solving and data comprehension skills.
- Strong communication skills and ability to work directly with stakeholders.