Real-time systems are seductive. You build a basic WebSocket connection, messages flow instantly, and it feels like magic. Then your first user's network hiccups, their browser crashes, or you deploy an update and suddenly 500 clients are screaming reconnect requests at your server. That's when you realize the Hello World examples glossed over the hard part.
I've built real-time collaboration features in CitizenApp, and I've learned that WebSocket architecture isn't about the connection itself—it's about what happens when the connection breaks. This post covers the patterns I actually use in production.
Why Connection Pooling Matters on the Server
Before we even talk about the client, let's address server capacity. Each WebSocket connection consumes memory and file descriptors. On my FastAPI backend, naive implementations spawn a new handler per connection and let garbage collection handle cleanup. That works until you hit 10k concurrent users and your CPU is thrashing.
I prefer explicit connection pooling with a bounded queue:
# FastAPI WebSocket Manager with pooling
from typing import Set
import asyncio
from fastapi import WebSocket
class ConnectionPool:
def __init__(self, max_connections: int = 5000):
self.active_connections: Set[WebSocket] = set()
self.max_connections = max_connections
self.lock = asyncio.Lock()
async def connect(self, websocket: WebSocket):
if len(self.active_connections) >= self.max_connections:
await websocket.close(code=1008, reason="Server capacity exceeded")
return False
async with self.lock:
self.active_connections.add(websocket)
return True
async def disconnect(self, websocket: WebSocket):
async with self.lock:
self.active_connections.discard(websocket)
async def broadcast(self, message: dict, exclude: WebSocket = None):
disconnected = set()
for connection in self.active_connections:
if exclude and connection == exclude:
continue
try:
await connection.send_json(message)
except:
disconnected.add(connection)
# Clean up dead connections
for conn in disconnected:
await self.disconnect(conn)
pool = ConnectionPool(max_connections=5000)
Why explicit pooling? Memory predictability. You know exactly how many connections you're holding. No surprise OOM kills at 3am.
The Heartbeat Pattern Prevents Zombie Connections
This one burned me hard. You think the connection is alive, but the client is offline or stuck. The server has no idea. Connections pile up consuming memory, and you're broadcasting to ghosts.
The heartbeat pattern (ping/pong) is your safety net:
// TypeScript Client with heartbeat
class ReliableWebSocket {
private ws: WebSocket | null = null;
private heartbeatInterval: number | null = null;
private heartbeatTimeout: number | null = null;
private messageQueue: any[] = [];
private reconnectAttempts = 0;
private maxReconnectAttempts = 5;
connect(url: string) {
return new Promise((resolve, reject) => {
this.ws = new WebSocket(url);
this.ws.onopen = () => {
console.log("Connected");
this.reconnectAttempts = 0;
this.startHeartbeat();
this.flushQueue();
resolve(this.ws);
};
this.ws.onmessage = (event) => {
const message = JSON.parse(event.data);
// Server heartbeat response
if (message.type === "pong") {
clearTimeout(this.heartbeatTimeout!);
return;
}
this.onMessage(message);
};
this.ws.onerror = reject;
});
}
private startHeartbeat() {
// Send ping every 30 seconds
this.heartbeatInterval = window.setInterval(() => {
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify({ type: "ping" }));
// If we don't get pong within 5s, reconnect
this.heartbeatTimeout = window.setTimeout(() => {
console.warn("Heartbeat timeout, reconnecting...");
this.reconnect();
}, 5000);
}
}, 30000);
}
private async reconnect() {
if (this.reconnectAttempts >= this.maxReconnectAttempts) {
this.onError(new Error("Max reconnect attempts exceeded"));
return;
}
this.reconnectAttempts++;
const backoffMs = Math.min(1000 * Math.pow(2, this.reconnectAttempts), 30000);
console.log(`Reconnecting in ${backoffMs}ms...`);
await new Promise(r => setTimeout(r, backoffMs));
try {
await this.connect(this.url);
} catch (error) {
this.reconnect();
}
}
private flushQueue() {
while (this.messageQueue.length > 0) {
const msg = this.messageQueue.shift();
this.send(msg);
}
}
send(message: any) {
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(message));
} else {
this.messageQueue.push(message);
}
}
private onMessage(message: any) {
// Your app logic here
}
private onError(error: Error) {
// Your error handling here
}
}
And on the server:
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
await pool.connect(websocket)
try:
while True:
data = await asyncio.wait_for(websocket.receive_text(), timeout=60.0)
message = json.loads(data)
if message.get("type") == "ping":
await websocket.send_json({"type": "pong"})
else:
await pool.broadcast(message)
except asyncio.TimeoutError:
await websocket.close(code=1000, reason="Heartbeat timeout")
except Exception as e:
print(f"Error: {e}")
finally:
await pool.disconnect(websocket)
Why? Heartbeats detect dead connections before they cause problems. No ghosts.
Graceful Reconnection with Exponential Backoff
Exponential backoff is non-negotiable. Hammering the server with reconnect requests during an outage creates a DoS attack on yourself.
Notice the Math.pow(2, this.reconnectAttempts) logic above. First attempt: 2s. Second: 4s. Third: 8s. Max out at 30s. This gives the network time to heal without overwhelming your server.
Message Queue During Disconnection
Users expect their messages sent even if briefly offline. The queue pattern above buffers messages and flushes when reconnected:
send(message: any) {
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(message));
} else {
// Queue for later
this.messageQueue.push(message);
}
}
Simple but critical. Without this, users see "sending..." spinners that never resolve.
Gotcha: Message Ordering During Reconnection
Here's what bit me: if the client reconnects while messages are still in flight, you can violate ordering guarantees. User A sends "create item", gets disconnected, then reconnects and sends "update item"—but the server's heartbeat timeout might have already processed the first message on a background task.
I solved this with idempotent message IDs:
private messageId = 0;
send(message: any) {
message.id = ++this.messageId;
message.timestamp = Date.now();
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(message));
} else {
this.messageQueue.push(message);
}
}
On the server, deduplicate:
seen_ids = set()
if message.get("id") in seen_ids:
return # Already processed
seen_ids.add(message.get("id"))
# Process message
This is boring infrastructure, but it's what separates "cool demo" from "production SaaS." Real-time systems fail constantly—your job is making those failures invisible to users.