About the role
- Must have:
- Scala development and design using Scala 2.10+ or Python
- Hadoop, Spark/Pyspark, Hive, YARN
- Data Modelling: A good data engineer should be able to design, implement and maintain data models that can support the organization's data storage and analysis needs, and demonstrate good use of associated tools
- Knowledge of operating systems such as windows and linux / unix
- Experience with Jira, Confluence, GitHub or other similar Agile Scrum technologies and automated deployment tools (Ansible & Jenkins)
- Knowledge and experience of test methodologies, developing test plans and execution of test cases (smoke/unit/integration/performance testing)
- Scheduling Pipelines using Control M/Airflow
- Should have:
- Work effectively within a pod structure, often unaided
- Attend daily scrums, meeting or exceeding sprint targets
- Providing L3 support for developed and deployed assets quickly and effectively, responding to tickets efficiently to minimise MTTR (Mean Time To Repair)
- Quick learner, Team player
- Nice to have:
- SQL, Jenkins, Cloud (GCP, AliCloud) , RESTful services
- Planning
- Work with Scrum Master to ensure:
- Tracking to Level 1 & Level 2 milestones using Jira
- Detailed sprint planning
- Ensure all communications cascaded as appropriate
- Validation
- Work with Product Owner and Lead Analyst to ensure Epics are technically sound
- Ensure Use Cases are fit for purpose for Data Engineer to code
- Review and signoff of detailed requirements with Lead Analyst
- Checklists
- Ensure Jira tickets maintained
- Ensure Confluence pages are maintained
- Work with the Scrum Master to ensure path-to-production checklists are complete
- Responsible to ensure all SNOW tickets raised to pod queue are managed efficiently
- Point of escalation for any technical-related query within the pod
- Teamwork
- Attendance and participation in Technical Product Manager Forum (TPM WG)
- Participation cross-pod
- Assistance with managing tribe technical capacity limitations ((production and pre-production environments)
- Advise and guide junior engineers within the pod
- Validate pod engineer’s work to ensure standards are maintained
- Escalate to tribe IT lead as required
Any Degree
- Teamwork
- Attendance and participation in Technical Product Manager Forum (TPM WG)
- Participation cross-pod
- Assistance with managing tribe technical capacity limitations ((production and pre-production environments)
- Advise and guide junior engineers within the pod
- Validate pod engineer’s work to ensure standards are maintained
- Escalate to tribe IT lead as required