About the role
JD:
- Must have:
- Scala development and design using Scala 2.10+ or Python
- Hadoop, Spark/Pyspark, Hive, YARN
- Data Modelling: A good data engineer should be able to design, implement and maintain data models that can support the organization's data storage and analysis needs, and demonstrate good use of associated tools
- Knowledge of operating systems such as windows and linux / unix
- Experience with Jira, Confluence, GitHub or other similar Agile Scrum technologies and automated deployment tools (Ansible & Jenkins)
- Knowledge and experience of test methodologies, developing test plans and execution of test cases (smoke/unit/integration/performance testing)
- Scheduling Pipelines using Control M/Airflow
- Should have:
- Work effectively within a pod structure, often unaided
- Attend daily scrums, meeting or exceeding sprint targets
- Providing L3 support for developed and deployed assets quickly and effectively, responding to tickets efficiently to minimise MTTR (Mean Time To Repair)
- Quick learner, Team player
- Nice to have:
- SQL, Jenkins, Cloud (GCP, AliCloud) , RESTful services