Scaling Real-Time APIs to 100k+ Concurrent Connections: WebSockets, SSE, and Redis Pub/Sub

Key Takeaway / TL;DR:

WebSockets and SSE serve different use cases: WebSockets for bidirectional, low-latency messaging; SSE for one-way server-to-client streams with automatic reconnection.

To scale beyond a single node, use Redis Pub/Sub as a message broker to fan out events across all connected clients, regardless of which node they're connected to.

Implement connection pooling, graceful backoff, and heartbeat mechanisms to maintain stability under high load and network instability.

The Problem & Industry Shift

Real-time features have become table stakes in modern applications: live chat, collaborative editing, stock tickers, and IoT telemetry. Traditional REST APIs with polling are inefficient and fail to meet latency requirements. The industry has shifted toward persistent connections using WebSockets and Server-Sent Events (SSE). However, scaling these connections to 100k+ concurrent users introduces significant challenges: connection state is tied to a single server, horizontal scaling requires a shared messaging layer, and network failures demand robust reconnection logic.

Architecture & Core Mechanics

WebSockets vs. SSE: A Technical Comparison

WebSockets: Full-duplex, message-based protocol over a single TCP connection. Ideal for bidirectional communication (e.g., chat, gaming). Requires a dedicated server implementation and handles reconnection manually.
SSE: Unidirectional, text-based event stream over HTTP. Simpler, with built-in auto-reconnection and event IDs. Ideal for live feeds, notifications, and server push updates.

Horizontal Scaling with Redis Pub/Sub

A single server can handle ~10k-50k connections, but beyond that, you need multiple nodes. The challenge: a client connected to Node A must receive events published by Node B. Redis Pub/Sub acts as a central message bus: each node subscribes to a channel, and when any node publishes an event, Redis broadcasts it to all subscribers, which then forward to their local clients.

+--------+     +--------+     +--------+
| Client |     | Client |     | Client |
+--------+     +--------+     +--------+
    |               |               |
    v               v               v
+--------+     +--------+     +--------+
| Node A |     | Node B |     | Node C |
+--------+     +--------+     +--------+
    |               |               |
    +-------+-------+-------+-------+
            |               |
            v               v
        +---------------------+
        | Redis Pub/Sub       |
        | (Message Broker)    |
        +---------------------+

Connection Pooling and Heartbeats

Connection Pooling: Reuse database connections and outbound HTTP connections to avoid overhead. For WebSockets, the OS handles many connections, but you must manage memory and CPU per connection.
Heartbeats: Send periodic ping/pong messages to keep connections alive and detect dead ones. For SSE, use comment lines (: ping) to prevent proxies from closing idle connections.

Production Code Example

Below is a Node.js implementation using ws for WebSockets and ioredis for Redis Pub/Sub. It demonstrates a scalable WebSocket server with Redis-based fan-out.

import { WebSocketServer, WebSocket } from 'ws';
import Redis from 'ioredis';

const wss = new WebSocketServer({ port: 8080 });

// Redis clients: one for publishing, one for subscribing
const pubClient = new Redis();
const subClient = pubClient.duplicate();

// Subscribe to the global channel
subClient.subscribe('global-events');
subClient.on('message', (channel, message) => {
  // Broadcast to all local clients
  wss.clients.forEach((client) => {
    if (client.readyState === WebSocket.OPEN) {
      client.send(message);
    }
  });
});

wss.on('connection', (ws) => {
  console.log('New client connected');

  // Send a welcome message
  ws.send(JSON.stringify({ type: 'welcome', data: 'Connected to server' }));

  // Handle incoming messages
  ws.on('message', (data) => {
    const msg = data.toString();
    // Publish to Redis so all nodes can broadcast
    pubClient.publish('global-events', msg);
  });

  // Heartbeat: send ping every 30s
  const interval = setInterval(() => {
    if (ws.readyState === WebSocket.OPEN) {
      ws.ping();
    }
  }, 30000);

  ws.on('close', () => {
    clearInterval(interval);
    console.log('Client disconnected');
  });
});

// Graceful shutdown
process.on('SIGINT', async () => {
  wss.close();
  await pubClient.quit();
  await subClient.quit();
  process.exit(0);
});

Key engineering decisions:

Use a dedicated Redis subscriber to avoid blocking the event loop.
Heartbeat interval of 30s balances liveness detection with network overhead.
Graceful shutdown ensures clean connection termination.

Performance, Cost & Trade-offs

Benchmarks

WebSockets: With Node.js and ws, a single node can handle ~50k concurrent connections with ~1ms latency on a 4-core machine. Scaling to 100k+ requires multiple nodes.
SSE: Simpler to implement, but each connection consumes an HTTP request slot. With HTTP/2 multiplexing, you can reduce overhead, but WebSockets are generally more efficient for high-frequency bidirectional traffic.

Redis Pub/Sub Overhead

Latency: Redis Pub/Sub adds ~0.1-0.5ms per message, which is negligible for most real-time apps.
Memory: Redis stores no message history; if a subscriber is down, messages are lost. For durable delivery, consider Redis Streams or a message queue like Kafka.
Cost: Running a Redis cluster adds operational overhead. Use managed services (e.g., AWS ElastiCache) to reduce maintenance.

Trade-offs

WebSockets vs. SSE: WebSockets require more complex client logic and server resources; SSE is simpler and works over standard HTTP, but is unidirectional.
Redis vs. Direct Node-to-Node: Direct WebSocket connections between nodes are complex and fragile; Redis provides a decoupled, scalable broker.

Actionable Checklist / Summary

Choose the right protocol: Use WebSockets for bidirectional, low-latency needs; SSE for one-way server push.
Design for horizontal scaling: Use Redis Pub/Sub as the message broker; each node subscribes to a channel and broadcasts to local clients.
Implement connection pooling: Reuse Redis clients and database connections to avoid resource exhaustion.
Add heartbeats: Use ping/pong (WebSockets) or comment lines (SSE) to keep connections alive and detect dead ones.
Implement graceful backoff: On client side, use exponential backoff with jitter for reconnection attempts to avoid thundering herd.
Monitor and tune: Track connection counts, message latency, and Redis CPU usage. Use load testing tools like k6 or wrk to validate performance.
Consider durability: If message loss is unacceptable, use Redis Streams or a message queue instead of Pub/Sub.