About the role
Requirements:
- 9+ years of backend development / distributed systems; Python or Go.
- Hands-on experience with one or more inference frameworks: vLLM, Triton, TensorRT-LLM, ONNX Runtime, PyTorch.
- Understanding of distributed training: data parallelism, tensor parallelism, pipeline parallelism, ZeRO.
- Knowledge of Kubernetes and cloud-native ecosystem; containerization and service orchestration.
- Proficiency in GPU cluster management fundamentals (CUDA, cuDNN, device plugins).
- Strong distributed systems design: load balancing, HA, resource scheduling.
Nice-to-Have:
- LLM serving at scale (vLLM, SGLang, TensorRT-LLM) in production.
- Quantisation techniques: INT8/FP8 sparsity, speculative decoding.
- Large-scale GPU training cluster operation (Slurm, Volcano, KubeRay).
- Training fault recovery, checkpoint management, elastic training.
- RDMA/NCCL/InfiniBand high-performance networking.
- Active open-source contributions (vLLM, SGLang, Megatron-LM, DeepSpeed, Ray).