Rust Engineer

LLM Gateway Infrastructure for AI Teams • San Francisco (On-site)

Location

San Francisco, CA (On-site)

Compensation

$230k–$290k + competitive equity

About the company

Our client is a venture-backed developer-infrastructure company in San Francisco. Their gateway sits between engineering teams and the large language models they build on — one interface that routes to every major model provider, so product teams can swap models without rewriting their stack. It is the kind of unglamorous infrastructure that everything else quietly depends on, and it is already profitable and growing fast.

The role

You'll own the core architecture of the gateway and lead its migration from Python to Rust — the single most consequential piece of engineering work at the company right now. You'll work directly with the CTO, define the technical roadmap for the rewrite, and solve real low-latency problems at a scale of over a billion requests a month. This is a small team, so the work is high-visibility by default: the reliability bar is enterprise-grade, and the person in this seat sets it.

What you'll do

Lead the migration of a production LLM gateway from Python to Rust, incrementally and without downtime
Own the core architecture and set the technical roadmap alongside the CTO
Drive down latency and raise throughput on a system serving over a billion requests per month
Build for enterprise-grade reliability — this is infrastructure other companies' products depend on
Work across the boundary between the existing Python services and the new Rust core

What we're looking for

These are hard requirements. Check each one that applies to you.

2+ years of professional software engineering experience
I've written Rust that ran in production — not tutorials, courses or side projects
Strong hands-on Python, and I'm comfortable moving code across the Python/Rust boundary
I've worked on low-latency, high-throughput backend systems where performance was the requirement rather than an afterthought
Production experience with PostgreSQL and Redis
I can work on-site in San Francisco
Already authorised to work in the US — this client does not sponsor new visas

Please note: every item above is a hard requirement for this role. Applications that don't meet all of them will be rejected. We recommend applying only if you check every box.

Don't check every box? No worries — submit your info here and we'll keep you in mind for roles that could be a better fit.

Submit Your Info

Nice to have

Experience with LLM APIs, inference routing or model-serving infrastructure
Prior work on a Python-to-Rust migration, or any incremental rewrite of a live production system
Open-source contributions to developer tooling or infrastructure projects
Early-stage experience where you owned architecture rather than inheriting it

Tech stack

Rust and Python, with PostgreSQL, Redis, FastAPI and the major LLM provider APIs.

Compensation & logistics

$230k–$290k base + competitive equity
On-site in San Francisco
Seed-stage, profitable, team of around 20
Three-stage interview process — no final-round panel
No new visa sponsorship; existing visa transfers may be considered

Back to all roles