How We Built City-Scale Systems: Architecture and Load

Why High-Load Systems Are Critical for Business

Building city-scale high-load systems is not just an IT task — it directly impacts business stability and financial performance. When thousands or millions of users rely on a platform at the same time, every architectural decision becomes a business decision. A slow checkout, a stalled government service portal, or a payment gateway that times out during peak hours translates into lost revenue, support costs, and damaged trust that takes years to rebuild.

In Uzbekistan this is no longer a hypothetical concern. Public e-government services, fintech apps, ride-hailing, delivery, and large e-commerce platforms now serve user bases comparable to the population of an entire city. The traffic is not evenly distributed: it spikes during paydays, public events, tax deadlines, and marketing campaigns. A system that comfortably handles an average day can collapse under a 10x spike if it was never designed for it. This article describes how we approach the architecture and load engineering of such systems — the principles, the trade-offs, and the mistakes we see most often.

Start With Load Models, Not Servers

The most common mistake we encounter is teams buying bigger servers before they understand their load profile. Hardware buys time, not correctness. Before writing a single line of infrastructure code, we build a load model that answers concrete questions: how many concurrent users at peak, what is the read-to-write ratio, how large is the payload, what is the acceptable latency at the 95th and 99th percentiles, and how spiky is the traffic over a day and over a year.

These numbers drive everything downstream. A system that is 95% reads (a news portal, a public registry lookup) is architecturally very different from one that is write-heavy (a payment ledger, an IoT ingestion pipeline). Conflating the two leads to either wasted money or systems that fall over under their actual workload.

• RPS at peak, not average — average traffic hides the moments that actually break systems.
• Latency targets per endpoint — a search query and a payment confirmation cannot share one SLA.
• Data growth rate — what is fast at 1 million rows can be unusable at 500 million.
• Failure cost — how much one hour of downtime costs the business in money and reputation.

Architecture Principles That Scale

City-scale systems are not monoliths scaled vertically until they stop fitting on one machine. They are designed from day one to scale horizontally — adding more identical nodes rather than bigger ones. This requires a few non-negotiable principles.

Statelessness at the application tier. Application servers should hold no session state in memory. Any request can be served by any node. State lives in shared stores — Redis for sessions and caching, the database for durable data. This is what lets you add or remove servers behind a load balancer without users noticing.

Asynchronous processing. Not everything needs to happen inside the request. Sending notifications, generating reports, processing media, reconciling payments — these belong in background workers fed by a message queue (Kafka, RabbitMQ, or a managed equivalent). The user gets an instant response; the heavy work happens behind the scenes. This single pattern absorbs enormous spikes by smoothing bursts into a steady queue the workers drain at their own pace.

Caching at every layer. The fastest query is the one you never run. A CDN serves static assets and cacheable responses close to users. An in-memory cache holds hot data. The database itself caches query plans and pages. The art is in invalidation — knowing precisely when cached data is stale — which is why we cache aggressively but always with explicit TTLs and clear invalidation rules.

Database as the usual bottleneck. In almost every high-load system we have built, the database is the first thing to strain. We address this with read replicas to spread read traffic, connection pooling (PgBouncer for PostgreSQL) to stop connection exhaustion, careful indexing, and — only when genuinely needed — partitioning or sharding. Sharding is powerful but adds permanent operational complexity, so we treat it as a last resort, not a default.

Designing for Failure

At city scale, failure is not an exception — it is a constant background condition. Disks fail, networks drop packets, a dependency times out, a deploy goes wrong. The question is never "will something fail?" but "what happens to users when it does?" Resilient systems degrade gracefully instead of collapsing.

We design with redundancy at every level: no single server, database, or availability zone whose loss takes down the whole platform. Circuit breakers stop a failing dependency from cascading — if the recommendation service is down, the product page still loads without recommendations. Timeouts and retries with backoff prevent one slow service from exhausting threads across the whole stack. Health checks and automated failover remove dead nodes from rotation before users hit them.

Observability and Load Testing

You cannot operate what you cannot see. Before a city-scale system goes live, it needs three pillars of observability: metrics (request rates, latencies, error rates, resource usage), structured logs, and distributed tracing that follows a single request across all the services it touches. When something breaks at 2 AM during a traffic spike, these are the difference between a five-minute fix and a five-hour outage.

Equally important is testing the system under realistic load before real users do. We run load tests that replay realistic traffic patterns — including the spikes — and we deliberately push past expected peaks to find the breaking point. Knowing your system breaks at 8x normal load when you only expect 3x is a comfortable position. Discovering it during a live national event is not. Chaos testing, where we intentionally kill nodes and inject latency in staging, validates that the resilience mechanisms actually work rather than just existing on paper.

• Define SLOs — explicit targets like "99.9% of requests under 300 ms" that make "fast enough" measurable.
• Alert on symptoms, not causes — page humans when users are affected, not on every CPU blip.
• Test the spike, not just the average — soak tests and burst tests reveal different failure modes.

Practical Recommendations for Business Owners

If you are commissioning or running a high-load system, a few priorities save the most money and pain. First, invest in the load model and architecture review early — fixing a scaling flaw in design costs a fraction of fixing it in production. Second, do not over-engineer for scale you do not have yet; build for one realistic step beyond current demand and design so the next step is possible, not pre-built. Premature sharding and unnecessary microservices have sunk more projects than they have saved.

Third, treat capacity as a continuous process, not a one-time launch task. Traffic grows, usage patterns shift, and a system tuned for last year's load will struggle with this year's. Regular load testing and capacity reviews keep you ahead of the curve. Finally, demand observability from day one — a system without good metrics and tracing is a black box that will eventually fail in ways no one can explain quickly enough.