About the role
Responsibilities:
- Own low-level solution design (LLD) for Spark/Scala pipelines, data services, and integration patterns.
- Lead, Design, develop, and maintain high-performance Spark and Scala data processing applications and pipelines on Amazon EMR.
- Ensure performance, reliability, and responsiveness of applications and distributed data workloads through profiling, tuning, monitoring, and capacity planning.
- Build streaming and event-driven data systems using Kafka and Pub/Sub patterns, including near-real-time processing where applicable.
- Lead, design, and optimise data models and queries across PostgreSQL and Couchbase/Document DB; improve SQL query performance and data access patterns.
- Lead end-to-end delivery for a squad: define technical tasks, remove blockers, drive implementation, testing, deployment, and operational readiness.
- Define and promote engineering best practices, including code reviews, automated testing, CI/CD, secure coding, test-driven development and agile methodologies.
- Mentor and coach engineers; facilitate technical grooming, knowledge sharing, and upskilling to improve team capability.
- Coordinate with Operations and Infrastructure/DevOps teams for deployment and production support; lead major incident response, root-cause analysis (RCA), observability improvements (logs/metrics/traces), and technical debt reduction.
Requirements:
- Bachelor's degree in computer science, Information Technology, or equivalent experience.
- 7+ years of professional software development experience with strong hands-on coding and demonstrated technical leadership for projects or small teams.
- 5+ years of hands-on experience with Apache Spark and Scala, including building, scaling, and optimising large-scale batch data processing pipelines.
- Proven experience with Amazon EMR, including configuration, tuning, orchestration, troubleshooting, and cost/performance optimisation of Spark workloads in a cloud environment.
- 5+ years of Hands-on experience developing cloud solutions (AWS preferred; Azure acceptable), including S3 and integration with cloud-native services.
- Working experience building distributed systems using Pub/Sub patterns and streaming platforms like Kafka.
- Proficiency with SQL and NoSQL databases (e. g., PostgreSQL and Couchbase/Document DB), including schema design and SQL query optimisation for application performance.
- Experience with containerization and orchestration (Docker, Kubernetes/OpenShift).
- Strong performance engineering skills (profiling, tuning, capacity planning) across Spark jobs, services, and data stores.
- Solid understanding of DevOps tools and practices (Git, JIRA, Harness/Jenkins, CI/CD pipelines) and automated testing.
- Strong communication and problem-solving skills; ability to translate technical decisions and trade-offs for cross-functional stakeholders.
Preferred Qualifications:
- Experience with enterprise integration patterns (e. g., Apache Camel).
- Experience with microservice architecture and frameworks (Spring Boot, JPA/Hibernate) and designing resilient distributed services.
- Experience with ReactJS or a similar modern JavaScript framework (full-stack exposure).
- Experience working with highly concurrent distributed systems (e. g., Akka-based applications).
- Domain experience in travel, lodging, pricing, or revenue management applications/services/operations.
- Exposure to enterprise/solution architecture and platform modernisation initiatives.
- Experience working in Agile/Scrum methodology, including facilitating technical ceremonies and release planning.
- Preferred experience in Java/JEE development; working knowledge of Spring Boot and building APIs/microservices.