We evaluated Go, Rust, and Python for our LLM proxy. Go won - and not for the reason you'd expect.
go
dev.to
We built our LLM proxy in Go. Not Rust. Not Python. Here's the engineering trade-off nobody talks about: the language that's fastest in benchmarks isn't always the language that ships the fastest product. TL;DR Go handles 5,000+ RPS with ~11 microseconds of overhead per request — more than enough for 99% of LLM proxy workloads. Rust is faster (sub-1ms P99 at 10K QPS), but the development velocity trade-off isn't worth it unless you're building for hyperscale. Python (LiteLLM) hits a wall at ~