How to Design a Scalable Backend: Microservices, Caching, Queues, and Database Sharding

1. The 5 Golden Rules of Scalable Backend Architecture

Designing a scalable backend is not about choosing the newest framework; it is about decoupling state from computation. A backend that scales from 1,000 to 1,000,000 concurrent requests adheres to five non-negotiable principles:

1. Stateless Application Servers: Never store user sessions, uploaded files, or mutable state on server disk. Store state in distributed caches (Redis) and object storage (S3) so server instances can scale horizontally behind a load balancer.

2. Asynchronous Task Processing: Decouple slow operations (email dispatch, PDF generation, AI model inference, payment reconciliation) using message queues (BullMQ / RabbitMQ / AWS SQS). User-facing HTTP handlers should return 202 Accepted in under 50ms.

3. Aggressive Caching (Cache-Aside): Prevent database read exhaustion by serving repetitive queries directly from an in-memory Redis cluster.

4. Database Connection Pooling: Use PgBouncer or serverless connection pooling to prevent thousands of simultaneous client connections from crashing the database.

5. Fault Tolerance & Circuit Breakers: Wrap all third-party API calls in circuit breakers with graceful fallbacks so an external outage does not cascade through your core services.

2. Scalability Bottlenecks and Proven Engineering Solutions

Every high-concurrency software application hits standard scaling bottlenecks as traffic grows:

3. Implementation Code: Cache-Aside Pattern in TypeScript

The Cache-Aside pattern ensures that 90%+ of repetitive read queries never hit your primary relational database:

// lib/cache/cache-aside.ts
import { Redis } from "ioredis";
import { prisma } from "@/lib/prisma";

const redis = new Redis(process.env.REDIS_URL!);

export async function getCachedUserData(userId: string) {
  const cacheKey = `user:profile:${userId}`;

  // 1. Check Redis In-Memory Cache (Latency: < 2ms)
  const cachedData = await redis.get(cacheKey);
  if (cachedData) {
    return JSON.parse(cachedData);
  }

  // 2. Cache Miss: Query PostgreSQL (Latency: 15–40ms)
  const user = await prisma.user.findUnique({
    where: { id: userId },
    select: { id: true, email: true, name: true, plan: true },
  });

  if (user) {
    // 3. Write Back to Cache with 10-Minute Expiration (TTL)
    await redis.setex(cacheKey, 600, JSON.stringify(user));
  }

  return user;
}