About the role
We're looking for a Senior Software Engineer (P4) to join our Data Foundation Platform team. We build and provide cloud platform and services that are used by data engineering teams and data scientists across Disney Entertainment and ESPN Technology, particularly in processing data related to our streaming and web-based businesses (e. g., data from Disney+, Hulu, ESPN, etc. ). We are looking for a rockstar engineer with experience building tools and services that help Data Engineering teams execute efficiently, namely metadata management, cost data management, lifecycle management, etc. The successful candidate will have a passion for technology and solid coding skills. Knowledge of specific technologies may be helpful, but it is less important than the ability to learn quickly. We value team members who aren't afraid to dive deep when required, think for themselves, and have great communication and teamwork skills.
Responsibilities:
- Contribute to maintaining, updating, and expanding existing Data platform projects written in Java, Spark, Python, and Scala.
- Tech stack includes Airflow, Spark, Databricks, Delta Lake, Snowflake, Opensearch, Neptune, and Graph QL.
- Architect, design, and code shared libraries that abstract complex business logic to allow consistent functionality across all data pipelines in the Data organisation.
- Collaborate with product managers, architects, and other engineers to drive the success of the Data platform.
- Contribute to developing and documenting both internal and external standards and best practices for configurations, naming conventions, partitioning strategies, and more.
- Ensure high operational efficiency and quality of the Data platform projects.
- Be an active participant and advocate of agile/scrum ceremonies to collaborate and improve processes for our team.
- Engage with and understand our customers, forming relationships that allow us to understand and prioritise both innovative new offerings and incremental platform improvements.
- Maintain detailed documentation of your work and changes to support data quality and data governance requirements.
Requirements:
- BS/BA in Computer Engineering, Computer Science, Information Technology, or a closely related field.
- 7+ years of software engineering experience in developing large projects.
- Strong algorithmic problem-solving expertise.
- Strong fundamental programming skills.
- Proficient in AWS cloud technology or any other cloud technology.
- Proficient in Data Engineering technologies like Airflow, Spark, Databricks, Snowflake and EMR.
- Strong scripting language experience.
Preferred Qualifications:
- Passion to explore new technology and participate in proof of concepts.
- Enjoy new and meaningful technology or business challenges that require you to think and respond quickly.
- Passionate to automation, data, technology, and creative innovation.
- Prior experience building internet-scale platforms.
- Experience with technologies such as Kubernetes, Kafka, Spark, Airflow, and AWS.
- It would be awesome if you have a robust portfolio on GitHub and/or open-source contributions you are proud to share.
- Bachelor's or master's degree in Computer Science or related discipline, or its equivalent.