About the role
Responsibilities:
- Design, build, and maintain scalable real-time data streaming pipelines.
- Work hands-on with Apache Kafka and/or Apache Flink for event ingestion and stream processing.
- Build and operate data platforms, reusable frameworks, internal services, and data infrastructure.
- Develop production-grade code using Java, Python, or Go.
- Design and optimize distributed data processing systems for scalability, reliability, and low latency.
- Implement monitoring, logging, alerting, automated testing, and failure recovery mechanisms.
- Troubleshoot streaming pipeline issues, data loss, processing delays, and performance bottlenecks.
- Collaborate with engineering, product, and analytics teams to support real-time data requirements.
- Ensure data quality, integrity, availability, and operational reliability.
Requirements:
- 3-8 years of relevant experience in data engineering, data platform engineering, backend infrastructure, or distributed systems.
- Hands-on Apache Kafka and/or Apache Flink experience in real projects.
- Experience building or operating real-time streaming systems; batch-only data engineering experience is not sufficient.
- Demonstrated ownership of a data platform, framework, ingestion system, or service.
- Strong coding skills in at least one of Java, Python, or Go.
- Understanding of distributed systems, event-driven architecture, scalability, and fault tolerance.
- Candidates should have built or operated production streaming systems, implemented Kafka/Flink-based data processing, and taken ownership of scalable platforms or reusable engineering frameworks.
Good to Have:
- Experience with Apache Spark or Spark Structured Streaming.
- Exposure to AWS, GCP, or Azure.
- Experience with data processing frameworks, workflow orchestration, and cloud-native infrastructure.
- Experience with high-throughput systems or fintech/product engineering environments.
- Knowledge of monitoring, observability, and data quality practices.