10,000+ CCU: Managing Traffic Waves on Kubernetes

For six years I've held the technical ownership of a multiplayer platform that runs 10,000+ concurrent users. The traffic isn't a flat line, and it's not a predictable curve: peaks between 8–11 PM, spikes on weekends, 2× multipliers on promo days, unknown waves during feature launches.

When I say "the system is always up" what I really mean is: none of those waves land visibly on the user. This post walks through the real pieces of the Kubernetes setup behind that.

CPU-only HPA isn't enough for a game backend

Kubernetes's default autoscaler watches CPU. You set "scale when CPU > 80%" and you're done. That's correct for a typical CRUD API. It's wrong for a multiplayer game backend.

Why: CPU is the last thing to heat up in our shape of load. Our real bottlenecks ordered from hottest to coldest:

1. Concurrent game room count
2. WebSocket connection count
3. Redis Pub/Sub fanout pressure
4. Database write throughput
5. ...
6. Finally CPU

By the time CPU hits 80%, room wait times are already up, players have started complaining, and churn has begun. CPU gives us the signal about three minutes late.

The fix: custom-metrics HPA

With Prometheus + prometheus-adapter, we wired HPA to our own metrics:

• game_rooms_active — active rooms per pod
• websocket_connections — active connections
• redis_pubsub_queue_depth — queue depth as a stability proxy

HPA targets "50 active rooms per pod." At 60, new pods come up; at 40 they scale down. CPU is nowhere in the loop — we're already scaling ahead of the real bottleneck.

That one change cut "felt slow" complaints by roughly 70%.

Pre-warming: from reactive to proactive

HPA takes 30 seconds to a minute to bring up a pod after it detects a need. Image pull, init, health check, then taking traffic. Long enough to be painful when a big wave starts.

Fix: time-based pre-scale. Our traffic curve is predictable — ramp starts around 7:30 PM. A cronjob at 7:25 raises minimum replicas from 3 to 8. When the ramp hits, pods are hot and ready. HPA takes over from there.

Sounds like a hack. In production engineering, whatever works is good.

Pod priority and eviction behavior

On a game platform, not all pods are equal. A pod holding a live game session is far more valuable than a pod doing background stats aggregation.

Kubernetes has PriorityClass exactly for this. Our game pods run as high priority, background work as low. Under node pressure, low-priority pods get evicted first; high-priority pods stay. Simple, critical.

Related: game pods run with a 60-second graceful shutdown. On termination, the pod first stops accepting new connections, drains existing sessions, then exits. A user never gets kicked out of a live match because of a pod rotation.

Cold start and the JVM

Java/Spring game servers have a special wrinkle on Kubernetes: the first 30 seconds, the JIT is warming up. HPA can bring up a new pod, but that pod doesn't become useful immediately.

Fix: the readiness probe doesn't just check the port is open — it checks an internal warm-up endpoint. Until the JVM is warm, the pod doesn't go "ready." Kubernetes holds traffic out while the HPA manages the queue.

Invisible layer, very valuable. Without it, you get "new pod spawned, traffic hit, felt slow, appeared to autoscale successfully but actually served badly for a minute" — a very sneaky regression pattern.

What a traffic spike actually looks like

Last month, a promo day. Expected traffic +50%, got +150%. First 10 minutes:

1. Rooms fill up — game_rooms_active climbs
2. HPA triggers, new pods come up
3. Nodes get full, cluster autoscaler provisions more
4. Redis Pub/Sub pressure rises — a separate HPA handles that layer
5. DB write pressure rises — we use a write-behind pattern that absorbs the burst

Total incidents: zero. At peak, ~40 extra pods were running; as traffic normalized they scaled back down automatically. Nobody did anything manual. That's a good scale story.

Takeaway

Using Kubernetes autoscaling is important, but it's not sufficient. The real key is knowing why your system becomes a bottleneck and wiring your scaling decisions to that signal. CPU isn't the real signal for most production workloads.

A system that's been up for a long time isn't running one automation rule — it's running many interconnected automation layers. 10k+ CCU isn't built overnight. It's built in layers, over five years.