Scaling WebSockets: Connections, State, Broadcast

HTTP requests are stateless. You can hit an API at 10,000 RPS and each request is independent — they spread across servers, scale horizontally, clean and easy.

WebSockets aren't from that universe. Each connection is persistent: the user connects, the server holds the socket, messages flow through it. 10,000 concurrent users = 10,000 open sockets. As a scaling problem, it's fundamentally different from HTTP.

We've run a multiplayer platform with 10,000+ concurrent WebSocket connections for 4+ years. The real lessons.

1. Sticky session or pub/sub — pick one

User A connected to pod-A. User B is on pod-B. They want to message each other. How does pod-A's user get a message from pod-B's user?

Two approaches:

Sticky sessions — all of a user's connections route to the same pod. Same room → same pod. Pod-internal broadcast suffices.

Issue: when a pod restarts, the whole room scatters. Horizontal scaling is also harder — you're tied to stickiness.

Pub/sub — pod count doesn't matter. Each pod subscribes to Redis Pub/Sub for relevant rooms. A message arriving at one pod publishes to Redis, all pods deliver to their connected users in that room.

We use this. Reasons: horizontal scaling works, and rooms survive pod restarts.

2. Connection state — pod-local or distributed

Each connection has memory cost: user id, room, time connected, last ping. 10,000 connections = ~50–100 MB of memory.

Two approaches:

Pod-local state — each pod knows its own connections, nothing more. Simple, fast.

Distributed state — global connection map in Redis or similar. Every pod knows where every user is.

We use pod-local + Redis Pub/Sub. Keeping a distributed state in sync is operationally expensive. We get by with pod-local because we're not sticky and pub/sub broadcast is enough.

For "is user X online?" questions we use a separate Redis set: online_users. Add on connect, remove on disconnect, periodic cleanup.

3. Heartbeat and dead-connection detection

WebSocket connections die silently. User's internet drops, NAT timeout, device powered off — server gets no notice. The pod thinks the user is still there, tries to send a message, fails or accumulates a zombie connection.

Solution: heartbeat.

Every 30 seconds the client sends a ping to the server. If 90 seconds pass with no ping, the server considers the connection a zombie and closes it.

Server-to-client pings work too. We prefer client-driven because it lets the server stay passive — less CPU work.

Bonus: enable TCP keepalive at the OS level. But default keepalive is 2 hours — useless for mobile clients. Application-level heartbeat is usually the real one.

4. Graceful shutdown

A pod needs to restart. 1,000 users connected. If the pod dies suddenly, 1,000 users get dropped.

Solution: graceful shutdown.

1. Pod gets shutdown signal (SIGTERM)
2. Stops accepting new connections (load balancer also drops it)
3. Sends a "reconnect please" message to existing connections
4. Users reconnect to other pods
5. Wait 30–60 seconds, force-close any remaining
6. Pod exits

In Kubernetes the terminationGracePeriodSeconds: 60 setting controls this. We use 90 seconds for game pods so mid-round matches can finish.

5. Broadcast scaling — pub/sub at scale

10,000 users subscribed to a global event (e.g., "today's leader changed"). The message has to reach all of them. Pub/sub fanout copies the message to 10,000 sockets — 10,000 sends.

In practice:

• Keep messages small (do you need binary instead of JSON?)
• Send asynchronously, in parallel
• Slow consumers — kick, drop, log

Don't let one slow consumer (a slow device) block the broadcast. When the buffer fills, drop the message and log it.

6. Authentication and hijacking

Once a WebSocket connection is established, "is this still actually user X?" gets harder. There's no token expiry mid-stream, no refresh.

Our approach:

• Validate the token on the initial handshake
• When the token expires, kick the connection and require reconnection
• Periodically (every 5 minutes) re-validate the token

This detail matters in production — especially for WebSocket flows that affect payments or permissions.

Scaling checklist

What I ask before putting a WebSocket system in production:

• Is connection state pod-local or distributed (and is the choice deliberate)?
• Pub/sub or sticky (deliberate)?
• Heartbeat in place? With what threshold?
• Graceful shutdown? With what duration?
• Slow-consumer drop policy?
• Per-pod connection limit? (memory + CPU bound)
• Client-side reconnection logic with exponential backoff?
• WebSocket-aware load balancer? (Layer 7, not naive NLB/ALB)

Closing

WebSocket scaling needs a different mental model from classic HTTP scaling. Once you switch to connection-as-resource thinking, it gets manageable. Where most teams trip is starting with HTTP-style assumptions and getting hit by the first traffic wave.

Our setup matured over five years. If I started a similar system today, this six-item list would be the day-one checklist.