tolgacelik.net
All posts
·3 min read

On-prem to AWS + Kubernetes: migrating a 20+ microservice iGaming platform

Notes from the 2023 rebuild of a high-traffic multiplayer platform serving 10,000+ concurrent users — polyglot stack, Istio mesh, and the decisions that actually paid off.

kubernetesawsmigrationistioigamingdistributed-systems

In 2023 I owned and technically led the rebuild of a multiplayer board-game platform from an on-prem datacenter into an AWS + Kubernetes cloud-native architecture. The system serves 10,000+ concurrent users, spans 20+ microservices across Java, .NET Core, PHP, Python, Node.js and React, and today is fully live on AWS.

This post is a honest retrospective: what we actually did, the decisions that were worth it, and the ones we would make differently.

The starting point

A polyglot platform built organically over years:

  • Java (Spring) for the core game servers and most of the long-lived backend services
  • .NET Core for several game-server variants and a set of API services
  • Python for async consumer services (matchmaking, payouts, fraud checks)
  • PHP (Laravel/Lumen) for legacy admin and tooling
  • Node.js for a handful of real-time facades
  • React + AngularJS + Vue across the various clients and operational panels

Data layer:

  • MySQL for transactional data
  • MongoDB for session state and flexible documents
  • Redis (Pub/Sub + HashMap) for real-time broadcast and hot caches
  • RabbitMQ for the async fabric
  • Memcached for read-heavy computed responses

This ran on bare-metal VMs with hand-crafted service orchestration. It was stable, but scaling meant calling the datacenter.

Why rebuild, not lift-and-shift

We had two options:

  1. Lift the current VMs into EC2 and keep going.
  2. Rebuild into Kubernetes + Istio with real isolation, autoscaling, and proper operational tooling.

Option 1 would have been faster for 3 months and slower forever after. We went with option 2 — not because "Kubernetes is cool," but because the new business requirements (live-ops campaigns, per-segment rollouts, regional latency) required capabilities that bare-metal VMs could not deliver without significant custom work. Kubernetes + Istio gave us those capabilities as first-class primitives.

What went right

Istio for service-to-service TLS and traffic shaping. Mutual TLS everywhere, free. Canary rollouts were a single VirtualService change instead of a reverse-proxy rebuild.

One Helm chart per service, versioned alongside the service. Dev teams owned their charts. No central platform team bottleneck on routine deploys.

Redis Pub/Sub stayed. We were tempted to "modernize" to a managed stream platform. We didn't. Redis Pub/Sub was battle-tested in our shape of traffic, and the migration itself was risky enough without swapping the broadcast fabric.

RabbitMQ stayed too. Same logic. Managed services on AWS are great, but "great" without our exact operational muscle around them was a net negative during migration.

What we would do differently

We underestimated the observability cost. The pre-migration system had bespoke dashboards per service and a shared Sentry. Kubernetes multiplies the surface area — pods come and go, IPs shift, logs fan out across containers. We had to retrofit a consistent logging and tracing convention mid-migration. Start with the observability story, not the compute story.

Per-service ownership of chart + CI/CD was right; per-service freedom of language runtime was wrong. Too many tiny variations. We standardized later but paid the debt.

What the system looks like today

  • All services on AWS EKS, behind Istio.
  • Horizontal pod autoscaling on custom metrics (concurrent game rooms, not CPU).
  • Blue/green at the routing layer, not the Kubernetes layer.
  • Sentry + ELK for observability.
  • CI via Jenkins, moving toward Actions.

The system is live. I still own its architecture, stability, and evolution.

If you're starting the same journey

Two pieces of advice:

  1. Rebuild the contract, not the code. The migration is an excuse to reshape what services promise each other. Use it. Don't port bad interfaces.

  2. Move the people, not just the workload. The ops skills for bare metal ≠ the ops skills for Kubernetes. Training and shadowing are more important than tooling choices.

Got a migration you're staring at? Drop me a note — happy to compare notes.

ShareLinkedInX
Get the next one in your inbox

Long-form notes on distributed systems, production stability, AI-assisted shipping. No spam, easy to unsubscribe.

Email only. Stored on this server. Never sold or shared.

Comments

No comments yet. Be the first.

Leave a comment

Never shown publicly. Only used if I need to reply.