Feature Flags: How They Actually Work in Production
Feature flags sound simple — "on/off switch." Done badly, they fill your codebase with zombie code over a year. Six years of production experience on what they're for and what they aren't.
A feature flag lets you turn a piece of code on or off at runtime. Activate without a deploy, deactivate without a deploy, target specific users — sounds great. It usually is great, but only with the right discipline.
Used wrong, feature flags can make a codebase unworkable in 6 months. Here's what I've learned.
When feature flags genuinely help
Three scenarios:
1. Rolling out risky changes gradually. You wrote a new payment algorithm. Deploy it, but it's hidden from all users. Open it to 5%. Watch metrics. Looking good? Move to 25%, then 50%, then 100%. Looking bad? Flip the flag off — code is still deployed, just inactive. Way faster than a rollback.
2. Product experiments. Product wants to A/B test two UX flows. Half the users get A, half get B. Measure conversion. Decide.
3. Emergency kill switch. A feature is causing trouble in production, you need it gone immediately. No time to deploy. Flip the flag, stop the bleeding. Fix at leisure later.
For these three, flags are extremely valuable. Outside these, be careful.
When feature flags become a trap
Adding a flag is easy. Removing one is hard. When you add a flag, you've added two branches to your codebase: flag-on and flag-off. Each has tests, maintenance, future interaction implications.
Three flags → 8 combinations. Five flags → 32. The test surface area explodes.
Traps:
- "Temporary" flags become permanent. "Let it canary for a week, we'll remove it" — then the team moves on, the flag is forgotten. Six months later no one knows what it does.
- Implicit flag dependencies. Flag B's behavior is different when Flag A is off. These dependencies are invisible in code review and explode in production.
- Combinations aren't tested. Tests for flag-on, tests for flag-off. Tests for the cross-product? Rarely.
Production discipline
Rules that matured over 6 years on our team:
Every flag has an expiration date
When you create a flag, open a JIRA ticket: "Remove this flag, deadline 2026-06-01." You're the owner. When the date arrives, your job is to remove it.
If the flag wasn't removed: either a permanent decision is made (the feature is now everyone's default) or it gets killed (the feature wasn't wanted, deleted with its code). Not "we might want it again" — that's how zombie code is born.
Per-user, not per-environment
Don't build flags as "on in dev, off in staging, half in prod." Flags are runtime decisions, not environment configurations. Otherwise you'll get the dreaded "works in staging, breaks in prod" pattern.
Per-user (or per-tenant) targeting: when a flag is on, what's the criteria — new users? beta group? 5% random? Be explicit.
Naming matters
Good: new_payment_algorithm_v2, enable_voice_chat, experimental_matchmaking_v3
Bad: flag1, temp_thing, feature_x
A reader six months later should understand what the flag does from its name alone.
Keep code paths minimal
Check the flag once at the top of the function, then branch. Don't sprinkle flag checks across 10 lines — combinations multiply.
# Good
def process_payment(payment):
if not feature_flag('new_algorithm'):
return process_payment_legacy(payment)
return process_payment_new(payment)
# Bad
def process_payment(payment):
validate(payment)
if feature_flag('new_algorithm'):
do_thing_a()
do_common()
if feature_flag('new_algorithm'):
do_thing_b()
finalize()
The bad version produces "half-old, half-new" behavior under flag flip-flops.
Tools
Two paths:
Third-party (LaunchDarkly, Flagsmith, Unleash): quick setup, polished UI, evaluation logs. Expensive, especially LaunchDarkly (per-user monthly pricing). Upside: invested, proven.
Roll your own: we have a simple Redis-backed system. Flags as key-value in Redis, an admin panel updates them, targeting rules in code. Free, but you maintain the tooling.
Small team: roll your own. Big team: pay LaunchDarkly. The line isn't sharp; it depends on your team's appetite for tooling work.
Closing
Feature flags are powerful, but they accumulate complexity. Disciplined, they speed you up; undisciplined, they're a disaster.
Every flag you add gets a deadline. When the deadline hits, remove it. That single rule will save you from the "flag nightmare" complaint six months later.
Flags are a tool for releasing features, not a place to stash code.
Long-form notes on distributed systems, production stability, AI-assisted shipping. No spam, easy to unsubscribe.
Comments
No comments yet. Be the first.