Horizontal Scaling

Horizontal Scaling (Scaling Out) is the process of adding more server instances to a system to handle increased load, rather than upgrading a single server to have more CPU or RAM (Vertical Scaling).

Overview

Vertical scaling (buying a bigger server) has a hard limit. Eventually, you cannot buy a computer with 10,000 CPU cores. Furthermore, a single server is a Single Point of Failure (SPOF). If it crashes, your entire API goes offline.

Horizontal scaling solves this by deploying multiple identical copies of your NestJS application across different servers. A Load Balancer sits in front of them, distributing incoming HTTP requests evenly among the instances.

Key Concepts

• Statelessness: The most critical requirement for horizontal scaling. Any single request must be able to be served by any instance.
• Session Stickiness: A load balancer feature that routes a specific user to the same server every time. This is an anti-pattern and should be avoided in modern APIs.
• Shared State: If instances need to share data (like user sessions or rate limit counters), they must use an external, centralized data store like Redis.

Code Examples

1. The Problem: Stateful Applications

If you store data in local memory (RAM), your app is stateful and cannot be horizontally scaled.

import { Injectable } from '@nestjs/common';

@Injectable()
export class ShoppingCartService {
  // VERY BAD: Local memory state
  private carts = new Map<string, any[]>();

  addItem(userId: string, item: any) {
    const cart = this.carts.get(userId) || [];
    cart.push(item);
    this.carts.set(userId, cart);
  }
}

Why this fails:
User hits Server A and adds a shirt to their cart. Server A saves it in its carts Map. User clicks “Checkout”, but the Load Balancer routes this request to Server B. Server B checks its local carts Map, sees it is empty, and tells the user their cart is empty!

2. The Solution: Externalizing State (Redis)

To make the application stateless, move the state out of the Node.js process and into a shared database.

npm install @nestjs/cache-manager cache-manager cache-manager-redis-yet

import { Injectable, Inject } from '@nestjs/common';
import { CACHE_MANAGER } from '@nestjs/cache-manager';
import { Cache } from 'cache-manager';

@Injectable()
export class ShoppingCartService {
  // GOOD: Using an external, shared state store
  constructor(@Inject(CACHE_MANAGER) private cacheManager: Cache) {}

  async addItem(userId: string, item: any) {
    const cacheKey = `cart:${userId}`;
    
    // Fetch from Redis
    let cart = await this.cacheManager.get<any[]>(cacheKey) || [];
    cart.push(item);
    
    // Save back to Redis
    await this.cacheManager.set(cacheKey, cart);
  }
}

Now, it doesn’t matter which server the Load Balancer routes the user to. Both Server A and Server B will look up cart:123 from the exact same central Redis instance.

Best Practices

• Stateless Authentication: Do not use in-memory session IDs. Use JWTs (JSON Web Tokens). A JWT contains the user’s identity encoded directly in the token. Any server can verify the JWT cryptographically without needing to look up a session in memory or a database.
• Background Jobs: If you have a @Cron job running in a horizontally scaled app (e.g., 5 instances), the cron job will execute 5 times simultaneously! You must move background jobs to a distributed queue (like BullMQ backed by Redis) to ensure only one instance picks up and executes the job.