Data developer

We are looking for a passionate and experienced Data Developer to join our engineering team. This role involves building scalable, real-time data streaming solutions and contributing to the development of modern data platforms using cutting-edge technologies.

Key Responsibilities:

• Design and implement real-time streaming pipelines using:
  • Azure Service Bus
  • Event Hubs
  • OR - Apache Kafka
  • Azure Functions
• Develop and optimize Spark Streaming applications, including:
  • Unbounded data processing
  • Streaming DataFrames
  • Checkpointing
  • Streaming joins
• OR - Build and deploy containerized microservices on Kubernetes, following best practices in cloud-native development.
• Work with modern open table formats such as:
  • Delta Lake
  • Apache Iceberg
  • Apache Hudi

Good to Have:

• Experience with observability stacks (Prometheus, Grafana, ELK).
• Hands-on experience with setting up and working with analytical data bases and query engines like Trino, StarRocks, Clickhouse e.t.c
• Experience on designing data products in data mesh architecture