Bridging QUIC and SIP: Building a Low-Latency Telephony Gateway in Rust

A low-latency media gateway in Rust: the client stream runs on QUIC, the carrier stream runs on UDP, and Tokio keeps the heavy CPU-bound transcoding work away from the async I/O threads.

The Problem & Why VoIP Needs a Unified Gateway

Many web-to-telephony platforms have a quiet design bottleneck: they rely on large WebRTC infrastructure that can be expensive to scale, and most teams accept the resource cost without question.

Traditional telephony calls commonly use the G.711 companding standard, which samples mono audio at 8kHz and represents each sample with 8 bits, producing a constant 64 kbps codec payload. The real network usage is higher after RTP, UDP, IP, and link-layer headers are included. In low-connectivity environments (such as satellite links, remote base stations, or cellular edges), maintaining that rate per call can be expensive and sensitive to network jitter.

The packet math makes the difference clear. With 20ms packetization, G.711 produces 160 bytes of audio fifty times per second. Add the minimum 12-byte RTP header, 8-byte UDP header, and 20-byte IPv4 header, and the rate is already about 80 kbps at the IP layer, before Ethernet, tunneling, encryption, or provider overhead. That is why I separate codec bitrate from actual network bandwidth when planning capacity.

If you are newer to digital signal processing: companding (compressing and expanding) is a legacy technique that maps linear PCM samples into 8-bit logarithmic representations. This saved wire bandwidth on early networks but limits traditional telephony audio to a narrow frequency range.

Deep learning speech models offer an alternative: Google's Lyra, a neural audio codec that uses generative speech models to compress speech to 3.2 to 9.2 kbps. That is the encoded speech bitrate reported by the Google project, not the full on-the-wire rate after transport overhead.

Lyra is the codec target for this architecture, but I need to be precise about the current repository. When prebuilt Lyra libraries are unavailable, build.rs compiles external/lyra/lyra_production_real.cpp. That file implements a custom experimental transform and quantizer; it does not use Google's Lyra model and should not be described as Google Lyra-compatible. The transport and FFI design can host the real codec, but the bitstream used by the default local test build is its own format.

The hard part is not only the compression. The hard part is moving it without breaking the latency requirements of real-time voice.

Browsers cannot speak SIP directly without a gateway, and legacy carriers do not understand modern QUIC-based protocols or application-specific speech codecs. To bridge this gap, I built a unified gateway in Rust. The REST control plane sits in backend, while the real-time transcoding and media forwarding engines sit in sip_gateway.

If you are newer to telephony: SIP (Session Initiation Protocol) is the signaling standard that handles call setup, ringing, and teardown - think of it as the control plane. RTP (Real-time Transport Protocol) carries the actual audio payload. Legacy carriers commonly rely on SIP and RTP. On the web side, QUIC is a multiplexed, encrypted transport built over UDP, and WebTransport is the browser API that gives web applications access to low-latency streams and datagrams over HTTP/3.

In plain terms, a gateway is a translator. It accepts modern, lightweight QUIC packets from a browser, decodes the negotiated speech payload, transcodes it into traditional logarithmic telephony bytes, and serializes it into RTP/UDP packets for the carrier. Symmetrically, it performs the reverse process for the incoming stream.

I built the gateway this way because I wanted to avoid the usual architectural trade-offs:

Not a heavy WebRTC signaling node that duplicates media state across the database path.
Not a generic proxy that stalls under heavy CPU loads.

Instead, the gateway is an ephemeral, stateful media engine. It authenticates users at the cloud edge, maps their active sessions in memory, transcodes their audio streams, and terminates connections cleanly when the carrier hangs up. It avoids database access in the per-packet media path, but it is not stateless while calls are active.

That sounds abstract until you look at the scheduling boundaries of real-time voice. A 20ms audio frame gives the system a tight processing budget. If CPU-heavy codec work runs inside the same runtime threads that poll network sockets, it can starve the executor and turn compute delay into jitter or dropped audio.

The 20ms frame duration is not a claim that users hear every single 20ms delay. It is one part of the engineering budget. Capture, packetization, encoding, queueing, network transit, jitter buffering, decoding, resampling, and playout all contribute to one-way latency. The goal is to keep every stage bounded because small delays accumulate across the complete path.

The result is a dedicated, decoupled media plane. That separation is the core of the gateway's design.

Rust, WebTransport, and the Telephony Gateway Architecture

The gateway is split into two halves with a strict boundary between them:

backend (Control Plane): An Axum service on Tokio with a SurrealDB-backed store. It manages user registrations, session mapping, and token authorization. It generates short-lived WebTransport JWT authentication tokens but never enters the per-frame media path.
sip_gateway (Media Bridge): The real-time media server. It hosts the QUIC server accepting WebTransport connections from clients, manages Media-over-QUIC (MoQ) tracks, binds to UDP sockets for legacy RTP streams, and handles native codec/PCM <=> G.711 conversion through C++ FFI wrappers.

At the container level, the system is a bidirectional routing bridge:

                      +-----------------------+
                      |     backend (Axum)    |
                      |  [Control & Sessions] |
                      +-----------+-----------+
                                  | JWT / API Keys
                                  v
+--------------+    QUIC      +------------------------+   SIP/UDP   +-----------------+
|  Web Client  |<------------>|      sip_gateway       |<----------->|   SIP Carrier   |
| (WT / MoQ)   | (WebTransport|                        | (RTP/G.711) |  (PSTN Network) |
|              |    & MoQ)    |    [Transcoding:       |             |                 |
|              |              |   Codec/PCM <-> G.711] |             |                 |
+--------------+              +------------------------+             +-----------------+

The gateway manages media session states via three core components:

MediaEngine: Holds codec configurations and sample rates (48kHz for the current web-side codec configuration, 8kHz for G.711).
MediaBridge: Coordinates active call media streams and owns the maps of stateful encoders and decoders.
RtpHandler: Binds to the local UDP port, deserializes incoming RTP packets, and serializes outgoing companded payloads.

The native codec selection deserves special attention. build.rs can link prebuilt libraries, but when those libraries are absent it automatically compiles the custom C++ wrapper and still enables the internal cfg(lyra) path. That means a successful build, or even a successful LyraEncoder::new, does not prove that Google's Lyra implementation was linked. A deployment that promises Google Lyra must identify the exact library and model artifacts at build time, verify bitstream interoperability against Google's encoder and decoder, and expose the selected codec implementation in readiness and metrics.

We keep the media forwarding plane in memory on purpose. Voice routing is hot state. Querying a database to check where to send the next 20ms audio packet would damage real-time latency. The control plane creates the session and authentication context, and the media plane routes active frames in memory.

Key Design Decisions

1. Offloading C++ FFI Transcoding to Tokio's Blocking Pool

The Problem: Executing native codec processing through C++ FFI is blocking work. A real neural codec can make that work especially expensive. Running it directly on Tokio's cooperative async workers can prevent those workers from polling network sockets on time.
The Solution: We delegate FFI transcoding tasks to Tokio's blocking thread pool using tokio::task::spawn_blocking. The async I/O loop reads the network data, extracts the buffer, and transfers ownership of the codec work to the blocking pool.
The Trade-off: Blocking tasks introduce scheduling and ownership overhead. More importantly, Tokio's blocking pool is not an unlimited CPU scheduler. At high call counts, the gateway still needs bounded admission or a fixed-size codec worker pool to prevent too many CPU-bound jobs from running at once.

If you are newer to this language: tokio::task::spawn_blocking takes a closure, runs it on a Tokio-managed blocking thread pool, and returns a JoinHandle future. It protects the cooperative async workers, but it does not create a permanently dedicated thread for every call. Once a blocking task has started, aborting its handle does not stop the native operation. The codec call must return by itself, which is another reason to bound the work and understand its worst-case execution time.

2. "Take, Execute, Put Back" Concurrency

The Problem: Neural codecs are highly stateful. They maintain internal history and synthesis state. Sharing an encoder instance across streams is incorrect, and locking a global codec registry map while native code runs creates severe contention.
The Solution: We implement a "Take, Execute, Put Back" pattern. When a frame arrives, the stream task briefly acquires a write lock, removes the encoder instance from the map, and immediately drops the lock. The task moves ownership of the encoder to the blocking thread. Once encoding is done, we re-acquire the lock briefly and insert the encoder back.
The Trade-off: This pattern requires two lock acquisitions per frame, but the critical sections contain only hash map operations. It also depends on frames for one stream being processed sequentially. If two frames for the same stream arrive concurrently, the second can observe that the codec has temporarily been removed and take a fallback path. A per-stream worker or mutex is the safer production evolution of this design.

// Remove the encoder to take exclusive ownership, dropping the lock immediately
let encoder_opt = self.encoders.write().await.remove(&key);

let (encoder_opt, encode_res) = if let Some(mut encoder) = encoder_opt {
    let encode_task = tokio::task::spawn_blocking(move || {
        let result = encoder.encode(&audio_frame);
        (encoder, result)
    })
    .await;

    match encode_task {
        Ok((encoder, result)) => (Some(encoder), Some(result)),
        Err(error) => {
            tracing::error!(?error, "Lyra encode task failed");
            (None, None)
        }
    }
} else {
    (None, None)
};

// Re-acquire lock briefly to put the encoder back
if let Some(encoder) = encoder_opt {
    self.encoders.write().await.insert(key, encoder);
}

3. Direct Logarithmic Companding in the RTP Layer

The Problem: Converting linear PCM samples to G.711 (mu-law or A-law) bytes during packet serialization can bottleneck throughput if it relies on unnecessary intermediate work.
The Solution: The RtpHandler uses direct bitwise mapping functions to convert samples during serialization.
The Trade-off: This couples companding logic directly to the RTP layer and still performs heap allocations. The current implementation creates both an RTP packet vector and a payload vector, so it is allocation-conscious, not zero-allocation. The next optimization is to reserve the final packet and write the payload directly after the header.

// In sip_gateway/src/transport/sip/rtp_handler.rs

let payload: Vec<u8> = match codec {
    CodecType::G711MuLaw => audio_frame
        .samples
        .iter()
        .map(|&s| linear_to_mulaw(s))
        .collect(),
    CodecType::G711ALaw => audio_frame
        .samples
        .iter()
        .map(|&s| linear_to_alaw(s))
        .collect(),
    _ => unreachable!(),
};

Companding the payload is only one part of correct RTP behavior. The current handler is intentionally narrow: it derives sequence information from the frame timestamp, writes a millisecond value into the RTP timestamp field, and uses a fixed SSRC. Carrier-ready RTP requires independent per-stream state, suitable random starting sequence numbers, timestamps, and SSRC values, and a G.711 timestamp that advances by 160 clock ticks for each 20ms packet at 8kHz. It also requires parsing or explicitly rejecting CSRC entries, header extensions, and padding, then using RTCP reports for loss, jitter, and synchronization.

4. Codec Fallback Must Be Negotiated

The Problem: Some current error paths send raw or simply mapped audio bytes when native codec creation or encoding fails. Those bytes are not automatically valid for the codec the track advertised.
The Solution: Put the codec implementation, codec identity, and payload version in the media envelope. If the selected codec becomes unavailable, either renegotiate a codec the peer supports or fail the stream clearly.
The Trade-off: Failing a call is visible, but silently changing the byte format under the same track is worse because it produces corruption that looks like a network or decoder problem.

5. Decoupling WebTransport Signaling from the Core Database

The Problem: Putting database operations inside the media path adds latency and gives every audio frame a dependency on persistent storage.
The Solution: Establish and authenticate the session first, keep active routing state in memory, and move persistent analytics or call records outside the per-frame path.
The Trade-off: In-memory state is fast, but it disappears when a node crashes. Important call events need a reliable asynchronous persistence strategy rather than untracked background tasks.

My production-hardening work covers protocol versioning, bounded codec workers, and proper load benchmarks. The current C++ FFI bindings run on CPU cores, so I measure and control the call capacity of a gateway node before considering hardware acceleration.

Deep Dives

1: Tuning UDP Sockets at the OS Edge

In high-concurrency telephony gateways, standard socket defaults may be insufficient. When processing many concurrent media streams, OS-level UDP receive queues can overflow during bursts, resulting in packet loss and audio stutter.

The current RtpHandler binds its socket directly with tokio::net::UdpSocket::bind. It does not yet configure larger SO_RCVBUF or SO_SNDBUF values with socket2. I treat that as deployment work to measure and tune, not as a performance feature that already exists.

The production socket setup configures the socket before handing it to Tokio:

use socket2::{Domain, Socket, Type};
use std::net::SocketAddr;

let socket = Socket::new(Domain::IPV4, Type::DGRAM, None)?;
socket.set_reuse_address(true)?;
socket.set_recv_buffer_size(2_097_152)?;
socket.set_send_buffer_size(2_097_152)?;

let address: SocketAddr = "0.0.0.0:40000".parse()?;
socket.bind(&address.into())?;

let std_socket: std::net::UdpSocket = socket.into();
std_socket.set_nonblocking(true)?;
let tokio_socket = tokio::net::UdpSocket::from_std(std_socket)?;

Buffer sizes must be verified on the deployed operating system because kernels can clamp or transform the requested values. Larger buffers absorb short bursts; they do not remove the need for fast consumers, packet-loss metrics, and an explicit overload policy.

2: QUIC and WebTransport Session Negotiation

In this gateway, WebTransport runs over HTTP/3 and QUIC. During connection setup, the browser and server establish QUIC with TLS 1.3 and negotiate HTTP/3 through ALPN. The client then creates a WebTransport session with an HTTP extended CONNECT request. The exact negotiation fields have changed between WebTransport draft versions, so the client, server, and h3-webtransport library versions must be tested together.

To manage authentication at the gateway edge, we parse the request URL query parameters to extract the JWT token generated by our Axum control plane. The gateway verifies the HS256 signature, expiration, and expected audience before accepting the session:

// Extract JWT from WebTransport CONNECT query string
let claims = match token {
    Some(t) => jwt::validate_gateway_token(t, &config)?,
    None => return Err(GatewayError::AuthenticationError("Missing token".into())),
};

Because query strings can appear in proxy or access logs, these short-lived tokens must be redacted from logs. URI-based bearer tokens are generally discouraged for exactly this reason. The current W3C WebTransport specification includes request headers, so my preferred path is an appropriate header when the target browsers support it. When compatibility requires a URL token, I make it short-lived and single-use, bind it to the intended call or session, and reject replay through its JWT ID. The split-based query parsing in the current gateway also needs a proper URL query parser so encoded values and duplicate keys are handled safely.

The gateway currently pins validation to HS256 and checks the expected audience. A stronger service boundary also requires issuer validation, required identity claims, key rotation, and asymmetric signing when the gateway should hold only a verification key instead of the backend's signing secret.

If the JWT is valid, the gateway accepts the WebTransport session and binds the validated identity to it. QUIC supports connection migration and NAT rebinding, but that protocol capability does not guarantee a seamless Wi-Fi-to-cellular handoff by itself. Browser behavior, path validation, timeouts, and application session state all matter. In the current code, transport.allow_spin(true) enables QUIC's spin bit; it does not enable migration, so migration behavior must be verified with integration tests.

3: Media-over-QUIC (MoQ) Pub/Sub Architecture

Once the WebTransport session is established, media is routed through moq-lite tracks. MoQ uses a publisher/subscriber model in which media is organized into tracks, groups, and objects:

Track: A continuous media source (for example, "audio-user_123").
Group: A collection of media objects and a useful subscription boundary.
Object: An addressable media unit containing bytes.

In the gateway, we publish each audio frame by creating and closing one producer group. The payload begins with an 8-byte little-endian timestamp, followed by the encoded audio bytes:

// In sip_gateway/src/transport/web/moq.rs

let mut group = producer.append_group();
let mut frame_data = Vec::with_capacity(8 + encoded_data.len());
frame_data.extend_from_slice(&timestamp.to_le_bytes());
frame_data.extend_from_slice(&encoded_data);

group.write_frame(Bytes::from(frame_data));
group.close();

This means the current implementation uses one group per audio frame, not one group per second of audio. Closing the group also does not automatically drop stale media. QUIC streams are reliable and ordered within a stream, so a real-time implementation still needs an explicit freshness policy using delivery timeouts, stream cancellation, datagrams, bounded queues, or timestamp-based discarding at the receiver. Current MOQT drafts define object and subgroup delivery timeouts for this purpose, but the application still has to configure and test them through the library version it deploys.

MoQ is still evolving as an IETF draft. The timestamp prefix is an application-specific format, so a stable wire contract needs an explicit version, codec identifier, sample rate, channel count, timestamp unit, and sequence number before clients depend on it.

4: Session Lifecycles on SIP Carrier Hangup

When a PSTN carrier hangs up an established call, it sends a SIP BYE. A CANCEL is used to stop an unanswered INVITE transaction before the call is established. Both messages contain the SIP Call-ID used for the dialog or transaction.

However, the internal registry (active_calls) maps calls by an internal call ID. If the server tries to process a hangup using the SIP Call-ID directly as the lookup key, the removal fails. The active call entry remains in memory, and the background media tasks can continue running.

To fix this crossover leak, the gateway scans active calls, resolves the SIP header to the correct internal registry key, aborts the tasks explicitly, and removes the call:

// In sip_gateway/src/transport/sip/server.rs

// Map incoming sip_call_id to internal call_id
let internal_call_id_opt = calls
    .values()
    .find(|c| c.sip_call_id == sip_call_id)
    .map(|c| c.call_id.clone());

if let Some(internal_call_id) = internal_call_id_opt {
    if let Some(call) = calls.get_mut(&internal_call_id) {
        if let Some(task) = call.web_to_pstn_task.take() {
            task.abort();
        }
        if let Some(task) = call.pstn_to_web_task.take() {
            task.abort();
        }
    }
    calls.remove(&internal_call_id);
}

This ensures that the gateway requests cancellation of both media tasks when a call is terminated. A secondary sip_call_id -> internal_call_id index removes the scan across active calls, and structured cancellation makes cleanup awaitable and verifiable.

The server also contains a SIP shutdown method that hangs up active calls, but the process currently handles Ctrl-C without calling and awaiting that method, and it does not yet wire the production SIGTERM drain path. My rolling-deployment sequence is to stop accepting new calls, mark the instance as draining, allow active calls to finish within a grace period, and then force cleanup. Receiving a signal is not graceful shutdown by itself; the resource lifecycle has to be completed before the process leaves.

What the Type System Enforces

Rust's type system helps us enforce architectural invariants at compile time:

Algebraic Data Types (ADTs): We use enums instead of raw strings to represent call topologies (WebToPstn, PstnToWeb, WebToWeb). This lets the compiler check exhaustive routing matches.
Ownership: Moving an encoder or decoder into spawn_blocking prevents simultaneous Rust access to that codec instance while the blocking operation runs.
Explicit Error Paths: Result makes setup, codec, and transport failures visible in the control flow.
FFI Safety Bounds: The wrappers manually declare Send and Sync with unsafe impl. This is a promise made to the compiler, not something the compiler proves about the C++ library. Each declaration must match the native library's real thread-safety guarantees.

Performance, Backpressure, and Bounded State

I build the gateway around explicit limits because unbounded behavior is where systems quietly become unreliable under load.

The current repository configures QUIC flow-control windows and datagram buffers, but it does not yet contain bounded per-stream audio channels. It also has a jitter_buffer_ms configuration value set to 100, but that value alone is not an implemented jitter buffer. The active stream maps are created with HashMap::new(), not pre-allocated to a measured call capacity.

These are the production controls:

Bounded Per-Stream Queues: Preserve frame order and apply a defined overload policy when codec workers fall behind.
Late-Frame Handling: Drop audio that has missed its playback deadline instead of allowing reliable delivery to build delay.
A Real Jitter Buffer: Reorder packets within a measured delay budget and expose late, lost, and discarded frame metrics.
Capacity Limits: Bound active calls, codec jobs, sessions, tracks, and memory per connection.

The repository currently has no Criterion benchmark suite proving a specific frame-processing time. I do not publish a latency number until it is measured on specified hardware with release builds, the exact native codec implementation named, realistic bidirectional calls, and latency percentiles.

I ran cargo test --lib against the current default build: all 40 tests passed. The build log also confirmed that prebuilt Lyra libraries were not found and the custom C++ wrapper was compiled instead. In its round-trip test, that wrapper encoded one 20ms, 960-sample frame to 248 bytes. At fifty frames per second, that is 99.2 kbps of codec payload, so this fallback does not demonstrate Google's 3.2 to 9.2 kbps bandwidth result. The test proves that the local encoder and decoder agree with each other; it does not prove Google Lyra bitstream compatibility, speech quality, packet-loss behavior, or production capacity.

The current resampler also uses simple linear interpolation between 8kHz and 48kHz. It is easy to follow, but production requires measured audio quality or a proper band-limited resampling library.

What I Measure Before Production

The gateway already exposes health and JSON metrics endpoints and keeps call-level metric structures. That is a useful start, but some values are placeholders or are not yet wired directly into the media path. Before production, I measure:

Codec queue time separately from native encode and decode execution time.
End-to-end frame age at p50, p95, p99, and maximum.
RTP packets sent, received, lost, late, duplicated, and reordered.
Jitter-buffer depth, late-frame discards, and concealment events.
MoQ objects expired, cancelled, dropped, or delivered too late to play.
Active calls, active codec jobs, Tokio task count, socket count, and memory per call.
Call setup time, teardown time, and leaked resources after repeated call churn.
Calls per core for each codec implementation and configured bitrate, with the CPU model and compiler flags recorded.

An average alone can hide the exact stalls that damage speech. For this kind of system, queue depth and tail latency tell me more than a single throughput number.

Scaling and Failure Boundaries

The media gateway keeps live call state in memory, so horizontal scaling is not just adding replicas behind a generic load balancer. New calls can be distributed across nodes, but all packets and control actions for an active call must reach the node that owns its SIP dialog, RTP socket, WebTransport session, MoQ tracks, and codec state.

In production, I make that ownership explicit:

Route a call to one gateway node and keep it there for the call's lifetime.
Advertise RTP addresses and ports that remain reachable for that node.
Stop assigning new calls before a node is drained or replaced.
Keep control-plane call records separate from media state so the backend can reconcile a node failure.
Accept that losing a media node normally drops its active calls unless the system implements a much more expensive state-transfer design.

This is why I call the gateway ephemeral and stateful. The node can be replaced, but an active call is still tied to the resources owned by that node.

Conclusion

By isolating native FFI codec work, keeping collection locks brief, and enforcing explicit lifecycle cleanup, the Rust telephony gateway creates a practical bridge between modern QUIC-based web clients and legacy SIP/RTP carriers.

The architecture is the strongest part of the system, but the implementation still needs bounded codec scheduling, RTP hardening, a real jitter buffer, a versioned media envelope, and reproducible load benchmarks before making production-scale performance claims.

The I/O loops own network scheduling. The blocking pool owns compute. The gateway owns resource boundaries.