About the role
Responsibilities:
- Build and maintain scalable ETL/ELT data pipelines for structured and unstructured data.
- Design and optimise batch and real-time streaming pipelines using Spark, Kafka, Flink, etc.
- Develop and maintain data models for analytics and product use cases.
- Write clean, scalable, production-grade code using Python, Java, Go, or Ruby.
- Build internal data services, integrations, automation, and developer tools.
- Ensure data quality, integrity, availability, reliability, and performance.
- Implement monitoring, logging, alerting, and automated testing for data pipelines.
- Troubleshoot pipeline failures and optimize performance, scalability, and cost.
- Collaborate with Product, Analytics, and Engineering teams on data requirements.
- Participate in system design and architecture discussions.
- Follow best practices around data governance, security, and compliance.
Requirements:
- 3-8 years of experience in data platform or data engineering roles.
- Strong proficiency in SQL and experience with relational and NoSQL databases.
- Hands-on experience with Airflow, DBT, Spark, or Flink.
- Experience with streaming or batch data processing systems.
- Proficiency in at least one programming language (Python, Go, or Java).
- Familiarity with cloud platforms (AWS, GCP, or Azure).
Nice to Have:
- Experience with real-time streaming (Kafka, Kinesis, Pub/Sub).
- Exposure to data warehouses (Snowflake, BigQuery, Redshift).
- Experience operating data platforms in high-scale or fintech domains.
- Understanding of data security, privacy, and compliance.