← All roles

[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering

Together AI ·San Francisco ·Onsite 9d ago
KubernetesGoPythonRustTemporalApache KafkaNATSCUDA

About the role

We are looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You will design a manifest-driven API, build controllers and reconciliation loops, and create a self-service API for the inference team. The role involves automating self-healing, defragmentation, and bin-packing to improve GPU utilization. Strong candidates have experience with Kubernetes controllers, platform APIs, and GPU scheduling.

Requirements

Core requirements: strong software engineering background in Go, Python, or Rust; experience with durable workflow orchestration (Temporal, Cadence); experience building software control planes with Kubernetes controllers; experience with event-driven systems (Kafka, NATS, SQS); product mindset. Nice-to-have: bare-metal provisioning, GPU cluster stacks, hyperscaler experience.

About the company

Together AI

Together AI is a leading AI company providing on-demand access to powerful AI models and accelerated compute infrastructure. They offer a comprehensive platform for inference, fine-tuning, and research, enabling developers and businesses to build and deploy cutting-edge AI applications efficiently and affordably.

View company page →

More from Together AI

Staff Platform Infrastructure Engineer
KubernetesAWSTerraformGoPythonTypeScriptApache KafkaHelmAmazon EKS
$240k–280k
Onsite · San Francisco
Staff Platform Infrastructure Engineer
$240k–280k
Onsite · San Francisco
KubernetesAWSTerraformGoPythonTypeScriptApache KafkaHelmAmazon EKS
Senior Software Engineer - Together Cloud Platform
GoKubernetesPostgreSQLAWSAzureGCPAmazon KinesisApache AirflowApache KafkaDistributed Systems
$160k–230k
Remote · Worldwide
Senior Software Engineer - Together Cloud Platform
$160k–230k
Remote · Worldwide
GoKubernetesPostgreSQLAWSAzureGCPAmazon KinesisApache AirflowApache KafkaDistributed Systems
Staff Engineer, Distributed Storage and HPC & AI Infrastructure
KubernetesGoPythonS3TerraformPrometheusGrafanaInfrastructure as Code
$250k–330k
Remote · Worldwide
Staff Engineer, Distributed Storage and HPC & AI Infrastructure
$250k–330k
Remote · Worldwide
KubernetesGoPythonS3TerraformPrometheusGrafanaInfrastructure as Code
Senior Software Engineer, Observability
PrometheusGrafanaClickHouseOTELGoPythonTerraformAnsibleHelmAWSGCPAzureDockerKubernetesPostgreSQLMongoDBRedisDistributed Systems
$200k–280k
Remote · Worldwide
Senior Software Engineer, Observability
$200k–280k
Remote · Worldwide
PrometheusGrafanaClickHouseOTELGoPythonTerraformAnsibleHelmAWSGCPAzureDockerKubernetesPostgreSQLMongoDBRedisDistributed Systems
Senior Software Engineer, Customer Insights
GoPythonJavaTypeScriptC++Distributed Systems
$180k–250k
Onsite · San Francisco
Senior Software Engineer, Customer Insights
$180k–250k
Onsite · San Francisco
GoPythonJavaTypeScriptC++Distributed Systems
$160k–280k
Onsite San Francisco
Posted 9d ago
Apply for this role
Opens job-boards.greenhouse.io ↗
About the company
Together AI
https://www.together.ai

Together AI is a leading AI company providing on-demand access to powerful AI models and accelerated compute infrastructure. They offer a comprehensive platform for inference, fine-tuning, and research, enabling developers and businesses to build and deploy cutting-edge AI applications efficiently and affordably.

View company page →