About the role
- Hands-on experience with setting up and working with real time streaming use cases using Azure Service Bus, Event Hubs, Kafka, Azure functions
- Hands-on experience with Spark streaming (Unbounded data, streaming data frames, Checkpoints, streaming joins e.t.c)
- Solid understanding of Kubernetes, containers, and microservices architectures.
- Experience with observability stacks (Prometheus, Grafana, ELK).
- Experience in working with different table formats like delta, iceberge and hudi(OTF - Open Table Format).
- Hands-on experience with setting up and working with analytical data bases and query engines like Trino, StarRocks, Clickhouse e.t.c
- Experience on designing data products in data mesh architecture