Senior Software Engineer I, Inference

Senior engineers are area owners who lead designs, raise engineering standards, and deliver measurable improvements to latency, throughput, and reliability across multiple services. They partner with product, orchestration, and hardware teams to evolve a Kubernetes-native inference platform and meet strict P99 SLAs at scale. Responsibilities include leading design reviews, defining SLIs/SLOs, implementing advanced optimizations (micro-batch schedulers, speculative decoding, KV-cache reuse), and,

IC3: ~3–5 years; IC4: ~5–8 years industry experience building distributed systems or cloud services. Strong coding in Python or Go (C++ a plus) and deep familiarity with networked systems and performance. Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks (Prometheus, Grafana, OpenTelemetry). Practical knowledge of inference internals: batching, caching, mixed precision (BF16/FP8), streaming token delivery. Proven track record improving tail latency (P95/P99