About the role
We are seeking a Senior Data Engineer with 4+ years of experience in building scalable, distributed, and high-performance data systems. In this role, you will architect, develop, and optimise real-time pipelines and analytical systems that enable Amazon teams to extract actionable insights and power intelligent decision-making across the organisation.
Responsibilities:
- Design, build, and maintain real-time and batch data pipelines for structured and unstructured datasets.
- Develop scalable data ingestion, transformation, and processing frameworks for high-volume data environments.
- Collaborate with product, analytics, and infrastructure teams to ensure data reliability, availability, and performance.
- Build APIs and data models that support large-scale analytics and machine learning applications.
- Optimise query performance, storage systems, and ETL workflows for massive datasets.
- Mentor junior engineers, contribute to best practices, and lead design reviews for critical data infrastructure.
Requirements:
- 4+ years of experience in data engineering, backend development, or large-scale data systems.
- Strong proficiency in Python; experience with Java or Scala is a plus.
- Hands-on experience with real-time processing frameworks such as Kafka, Spark Streaming, or Flink.
- Solid understanding of data modelling, distributed systems, and API integrations.
- Proficiency with SQL and NoSQL databases and working experience on AWS (S3 EMR, Redshift, Glue, Kinesis).
- Familiarity with web scraping frameworks (Scrapy, Selenium, Playwright) or similar data acquisition tools.
- Experience in optimising ETL pipelines and handling large-scale datasets.
Preferred Qualifications:
- Experience in AI/ML data pipelines or analytical dashboarding.
- Exposure to workflow orchestration tools (Airflow, Step Functions, or similar).
- Experience mentoring engineers or leading small technical teams.