[!IMPORTANT] This is a classic backend system design question because it combines distributed consistency, safe rollout, failure handling, and operational visibility.
🧭 At a Glance
| Area | What To Say |
|---|---|
| Core idea | Store config centrally, version every change, and let servers reload config dynamically without restart. |
| Push mechanism | Watch API, long polling, WebSocket, Redis Pub/Sub, Kafka, etcd, Consul, or ZooKeeper watches. |
| Consistency | Track which server applied which version. Use eventual consistency for normal config and two-phase rollout for critical config. |
| Safety | Validate config, support staged rollout, rollback, local cache, audit logs, and failure alerts. |
📌 Real Interview Prompt
Question: How would you design a system to ensure configuration updates are applied consistently across all servers without restarting them?
✅ Short Answer
I would build a centralized dynamic configuration service. Configs are stored centrally and every update creates an immutable version. Application servers keep the current config in memory and watch for new versions. When a new version arrives, each server fetches it, validates it, and atomically swaps the in-memory config. The config service tracks server-wise status such as applied, pending, and failed. For safety, I would add rollback, staged rollout, audit logs, local cache, and polling as a fallback.
🏗️ High-Level Design
Admin / CI-CD
|
v
Config Service
|
v
Config Store: etcd / Consul / ZooKeeper / DB
|
v
Application Servers watch + fetch + validate + atomic swap
💬 Expandable Q/A
How do servers apply config without restart?
Each server reads config through a config manager. When a new version is validated, the server creates a new immutable config object and atomically swaps the old in-memory reference with the new one. Existing requests can finish on the old config while new requests use the new version.
How do you know all servers have the same config?
Every server reports its applied version to the config service. The dashboard can show server-1 applied v42, server-2 pending v41, and server-3 failed v42 with a reason.
Can all servers switch at exactly the same time?
Usually eventual consistency is enough. For critical config, use a two-phase flow: PREPARE the new version, wait for servers to validate and reply READY, then send ACTIVATE.
What if the config service is down?
Servers should continue with the last known good config from memory or local disk cache. When the service recovers, servers reconnect and check the latest version.
⚠️ Common Mistakes
- Reading config only at startup.
- Not versioning config updates.
- No validation before rollout.
- No server-wise acknowledgement tracking.
- No rollback or local cached config.
📝 Final Summary
A strong answer includes centralized storage, watches or polling, immutable versions, atomic in-memory swap, server acknowledgements, staged rollout, rollback, validation, and graceful behavior when the config service is unavailable.