About the role
The Inference team builds and operates CoreWeave's Kubernetes-native inference platform for low-latency, high-throughput AI workloads. As a Staff Software Engineer (IC5), you will lead architecture, performance, and reliability across services, focusing on distributed systems, Kubernetes, inference optimization, and cross-team initiatives.
Requirements
8-12+ years building large-scale distributed systems, strong programming in Go/Python/C++, deep Kubernetes expertise, experience with low-latency high-throughput systems, inference systems (batching, caching, memory optimization), metrics-driven performance improvement. Preferred: inference frameworks (vLLM, Triton, etc.), GPU optimization (CUDA, NCCL), multi-team leadership.
About the company
CoreWeave
CoreWeave is a cloud provider specializing in an AI-native platform built to power complex AI workloads. They offer GPU and CPU compute, storage, and infrastructure control solutions, positioning themselves as the essential cloud for AI innovation and significantly reducing total cost of ownership.
View company page →More from CoreWeave
Senior Productivity Platforms Developer
PythonGoKubernetesTerraformAnsibleDevOps
$139k–185k
Onsite · Dallas, TX, Bellevue, WA, Sunnyvale, CA
Senior Productivity Platforms Developer
$139k–185k
PythonGoKubernetesTerraformAnsibleDevOps
Senior Software Engineer, Network Development
PythonGoLinuxKubernetesAnsiblePrometheusGrafanaHPC
$165k–220k
Onsite · Livingston, NJ / New York, NY / Sunnyvale, CA / Bellevue, WA
Senior Software Engineer, Network Development
$165k–220k
PythonGoLinuxKubernetesAnsiblePrometheusGrafanaHPC
Engineering Manager, Launch & Sandboxes
GoPythonKubernetesDistributed SystemsHPC
$165k–242k
Onsite · Livingston, NJ / New York, NY / San Francisco, CA / Sunnyvale, CA / Bellevue, WA
Engineering Manager, Launch & Sandboxes
$165k–242k
GoPythonKubernetesDistributed SystemsHPC
Software and AI Engineer
TypeScriptPythonGoJavaC#C++ReactNode.jsFastAPILangChainGitAWSAzureGCPDocker
$120k–160k
Onsite · Sunnyvale, CA / Bellevue, WA / Dallas, TX
Software and AI Engineer
$120k–160k
TypeScriptPythonGoJavaC#C++ReactNode.jsFastAPILangChainGitAWSAzureGCPDocker
Senior Security Engineer, Vulnerability Management
PythonGoRustLinuxKubernetes
$165k–242k
Onsite · Livingston, NJ / New York, NY / Sunnyvale, CA / Bellevue, WA
Senior Security Engineer, Vulnerability Management
$165k–242k
PythonGoRustLinuxKubernetes
Similar roles
[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering
$160k–280k
[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering
$160k–280k
Senior AI Serving Engineer, Backend
$190k–250k
Senior AI Serving Engineer, Backend
$190k–250k
Research Engineer, Post-Training Inference
$200k–290k
Research Engineer, Post-Training Inference
$200k–290k
Staff Machine Learning Engineer, ML Infrastructure
$184k–245k
Staff Machine Learning Engineer, ML Infrastructure
$184k–245k
Forward Deployed Engineer, Inference
$240k–415k
Forward Deployed Engineer, Inference
$240k–415k