Performance Engineer (Inference)

Location: Noida or Bengaluru
About the Role
Serving models efficiently is its own discipline, distinct from training them. As an Inference Runtime Engineer, you'll build the serving infrastructure that keeps GPU utilization high and cost per token low across everything we run, from rollout generation to production model serving.
What You'll Do
Build and optimize model-serving infrastructure: KV-cache management, continuous batching, and speculative decoding
Implement quantization (FP8/INT4) and intelligent routing to reduce serving cost per token
Support low-latency rollout serving for both internal model training and customer-facing workloads
Diagnose and resolve GPU utilization bottlenecks in production serving
Work with product engineering to expose serving capabilities through customer-facing APIs
What You'll Bring
5+ years of experience in systems or ML infrastructure engineering
Strong experience with model-serving systems (vLLM, SGLang, TensorRT-LLM, or comparable)
Hands-on CUDA or Triton experience
Solid understanding of quantization and batching strategies for inference
Proficiency in C++ and Python
Education
B.Tech in Computer Science or a related field; M.Tech a plus for senior scope.
Nice to Have
Experience at an inference-focused AI infrastructure company or a hyperscaler's model-serving team
Exposure to speculative decoding or other latency-reduction techniques in production

Work arrangement
Hybrid