Hire inference engineers who've actually shipped serving.
Inference engineers make large models fast and affordable to run. Few people have done it in production, and their titles rarely say so. HiringEye finds them by their work and vets each one with an AI voice interview, a coding assessment and an expert review before you meet them.
What a strong inference engineer does
They own the path from a trained model to a fast, reliable, cost-efficient endpoint: choosing the serving stack, tuning batching and memory, and trading latency against throughput and GPU spend.
WHAT WE LOOK FOR
- Production serving with vLLM, SGLang, TensorRT-LLM or Triton Inference Server
- Continuous batching, KV-cache management and memory planning
- Quantization (FP8, INT8, AWQ, GPTQ) without breaking quality
- GPU profiling and latency vs throughput trade-offs (p50, p99, tokens per second)
- Autoscaling and cost optimisation for inference clusters
- Observability, SLOs and safe model rollouts
Most great inference engineers are titled “ML Engineer” or “Software Engineer”.
Open-source contributions
Merged pull requests to vLLM, SGLang, TensorRT-LLM and related serving projects.
Benchmarks and write-ups
Engineers who publish serving benchmarks or performance deep-dives.
Talks and research credits
MLSys and PyTorch meetup speakers, and arXiv acknowledgments.
Warm introductions
Referrals from engineers who've shipped serving stacks with them.
AI runs Round 1. Our people make the call.
Latency regression scenario
p99 doubled after a model upgrade: how do they find and fix it?
Prefill vs decode reasoning
Where the time goes, and which levers move each.
Capacity planning
Sizing GPUs for a target QPS and budget, and defending the trade-offs.
Hiring inference engineers: questions
How long does it take to hire an inference engineer in India?
With HiringEye, the first ranked shortlist usually arrives within 48 hours of a locked brief. Time to an accepted offer then depends mostly on notice periods, which in India are commonly 30 to 90 days.
What skills should an inference engineer have?
Production experience with a serving stack such as vLLM, TensorRT-LLM or Triton, plus batching and KV-cache management, quantization, GPU profiling, and the judgment to trade latency against throughput and cost.
Where are inference engineers based in India?
Mostly in Bengaluru and Hyderabad, with growing pools in Pune and remote. Many work inside GCCs and AI startups under generic titles like ML Engineer or Software Engineer, which is why keyword searches miss them.
Need an inference engineer? Meet the shortlist in 48 hours.
Tell us the model, the stack and the scale. Pay only when your hire joins.