About the role
Requirements:
- Strong programming skills in Python and SQL.
- Experience with data processing leveraging programming skills in Python and Spark.
- In-depth understanding of the Kafka platform for (real-time) data ingestion and processing of high-volume data.
- Design and architect data flows and data management in a Cloud environment that are scalable, repeatable, and eliminate time-consuming steps.
- Clear skills in using version control systems like GIT and developing (technical) documentation (e. g., in WIKI).
- Experiences with infrastructure components as code for robust deployment, replication, and uniform management.
- Effective collaboration and communication skills to engage with cross-functional teams.
- A proactive mindset with the ability to drive tasks to completion.
- Proficiency in Agile/Scrum/Kanban methodologies for efficient product delivery and management.
- Familiarity with AWS cloud services like Glue, Athena etc.
- Bachelor's or Master's degree in Computer Science/Information Systems.
- At least 4+ years of experience in building data flow and data management on a modern big data tech stack.
- Strong experience in using ETL frameworks (e. g., Airflow, Jenkins) to build and deploy production-quality ETL pipelines.
- Knowledge of data structures.
- Open to learning and implementing new technologies.
- Ability to think and perform data engineering workstreams with a product mindset.
- Fluency in English.