Design Live Comments

“Design live comments” (Facebook Live, YouTube Live, Twitch chat) tests whether you understand real-time fan-out at scale. A single popular stream can have millions of concurrent viewers, each posting and reading comments in real time. The read side — pushing a firehose of comments to a huge audience — is the whole challenge.

1. Understanding the Problem

Functional Requirements

• Post a comment on a live stream
• See new comments in real time as they are posted, in order
• Load recent history when you join a stream that is already in progress
• Viewer count (a nice-to-have that uses the same infrastructure)

Non-Functional Requirements

• Low latency — a comment should reach other viewers in under ~1 second
• Massive read fan-out — one comment may need to reach millions of viewers
• Availability over consistency — it is fine if a viewer misses the occasional comment on a flooded stream; the stream must stay up
• Elastic — streams range from 10 viewers to 10 million; the system must scale per-stream
• Scale — 1M concurrent streams, a hot stream with 10M concurrent viewers, thousands of comments/second on that one stream

Back-of-the-envelope Estimation

Concurrent streams:        1M
Viewers on a hot stream:   10M concurrent
Comments on hot stream:    ~2,000 / second (a fraction of viewers post)
Fan-out on hot stream:     2,000 comments/s × 10M viewers
                           = 2 × 10^10 messages/s if sent to everyone (!)
Comment size:              ~200 bytes

That fan-out number is the tell: naively pushing every comment to every viewer is impossible. The design must bound what each client receives, not just scale the plumbing.

2. Core Entities and APIs

Data Model

-- Comments (Cassandra - partitioned by stream_id, time-ordered)
CREATE TABLE comments (
    stream_id    UUID,
    comment_id   TIMEUUID,     -- time-ordered id, also the sort key
    user_id      UUID,
    text         TEXT,
    created_at   TIMESTAMP,
    PRIMARY KEY (stream_id, comment_id)
) WITH CLUSTERING ORDER BY (comment_id DESC);

Partitioning by stream_id keeps a stream’s comments together and ordered by time — exactly the access pattern (fetch recent comments for one stream). A TIMEUUID gives a globally unique, time-sortable id without a central counter.

API Design

# Post a comment (usually sent over the WebSocket, shown here as REST)
POST /api/v1/streams/{stream_id}/comments
Body:   { text: string, client_comment_id: UUID }   # client id for dedup
Response: { comment_id, created_at }

# Load recent history on join (REST)
GET /api/v1/streams/{stream_id}/comments?before={comment_id}&limit=50
Response: { comments: [...], has_more: boolean }

# Subscribe to live comments (WebSocket)
WS  /api/v1/streams/{stream_id}/live
  → server pushes: { comment_id, user_id, text, created_at }

On connect, a client does two things: it loads recent history over REST (so it isn’t staring at an empty box), then subscribes over WebSocket for everything new from that point forward.

3. High-Level Design

The path a comment takes is persist → publish → fan out:

Write Service — receives a new comment, validates it, writes it to the comment store, and publishes it to the stream’s pub/sub channel. It returns quickly; delivery is asynchronous.

Comment Store (Cassandra) — the source of truth and the backing for history/backfill. Write-heavy and time-ordered, which fits Cassandra well.

Pub/Sub (Redis / Kafka) — one logical channel per stream (stream:{id}). The write service publishes once; every fan-out server subscribed to that stream receives it. This decouples the (small number of) writers from the (potentially huge number of) connection servers.

WebSocket Fan-out Servers — hold the viewers’ live connections. Each server subscribes to the channels for the streams its connected viewers are watching, and pushes new comments down those sockets. A single server handles tens of thousands of connections; a hot stream is served by many of them.

History/Backfill Service — serves the REST GET comments endpoint from Cassandra for late joiners and scroll-back.

4. Deep Dives

Deep Dive: Real-time delivery with WebSockets + per-stream pub/sub

Why not have the write service push directly to every viewer? Because the write service has no idea which of the hundreds of WebSocket servers each viewer is connected to, and coupling them would not scale. Instead we use a pub/sub channel per stream (the same pattern as the Real-time Updates lesson).

1. Viewer’s client opens a WebSocket to a fan-out server (chosen by the load balancer).
2. That server SUBSCRIBEs to stream:{id} if it isn’t already (it subscribes once per stream, not once per viewer).
3. A new comment is published to stream:{id}; every subscribed fan-out server gets one copy and pushes it to each of its local viewers of that stream.

The number of pub/sub messages is proportional to (comments × fan-out servers), not (comments × viewers) — a massive reduction. The last hop (server → its local viewers) is a cheap in-memory loop over sockets.

WebSocket is the right transport here because it is a persistent, low-overhead push channel. (SSE would also work since comments are one-directional from server to viewer, with posting done over a separate request — a reasonable alternative worth mentioning.)

Deep Dive: Surviving a viral stream

A stream with 10M concurrent viewers cannot be served by one connection server. Two techniques keep it tractable:

1. Shard viewers across many fan-out servers. 10M viewers / ~50K connections per server ≈ 200 servers for that one stream. Each subscribes to stream:{id} and serves its slice of viewers. Adding capacity for a hot stream is just adding more subscribers to its channel. To avoid a single Redis node becoming the bottleneck for a mega-stream, the channel itself can be sharded (e.g. stream:{id}:{shard}) with the write service publishing to all shards, or a tiered relay (a small set of “root” subscribers that re-broadcast to leaf servers) can be used.

2. Sample and aggregate comments. No human can read 2,000 comments/second, and no client should try to render them. On a hot stream, don’t deliver every comment — deliver a representative sample (say 20–50 per second), prioritising comments from friends, verified users, or those being liked. This caps per-client bandwidth regardless of how crazy the stream gets. The full stream is still persisted for history and moderation; it is only the live push that is sampled.

This is the key insight the interviewer is looking for: the problem isn’t moving bytes, it’s that the firehose exceeds what a human can consume, so you bound it deliberately.

Deep Dive: Ordering, late joiners, and history

Ordering. Comments carry a time-ordered comment_id (TIMEUUID). Clients render by that id, so even if two comments arrive slightly out of order over the network, they are displayed consistently. Perfect global ordering across a flooded stream is neither achievable nor necessary — “roughly chronological” is what viewers expect.

Late joiners / backfill. When you join a stream mid-broadcast you want to see the last few comments, not an empty box. The client first calls GET /comments?limit=50 (served from Cassandra), renders those, then subscribes over WebSocket for everything newer. A small overlap window plus client-side dedup on comment_id prevents gaps or duplicates at the seam between history and live.

Storage. Comments are append-only and are rarely read after the stream ends, so Cassandra with a TTL (e.g. keep comments for 30–90 days) controls cost. A separate moderation pipeline consumes the comment stream to filter spam and abuse — important at this scale, and easier to reason about as its own async consumer.

Viewer count. The count reuses the fan-out tier: each WebSocket server already knows how many of its sockets are watching each stream, so every few seconds it writes its local count to Redis (e.g. a hash viewers:{stream_id} with one field per server, or a key per server with a short TTL so a crashed server’s count expires). The displayed number is the sum across servers, read and pushed to viewers every few seconds like any other update. It is approximate and a few seconds stale, which is fine: nobody needs to know whether 10,000,000 or 10,000,412 people are watching.

5. Key Trade-offs

• Transport: WebSocket vs. SSE. WebSockets give one persistent channel that can carry both posting and receiving. The cost is stateful connection servers and load balancers that must handle protocol upgrades and long-lived connections. Because comments flow one way to viewers and posting can go over a normal request, SSE over plain HTTP (with built-in reconnect) is a perfectly good choice, particularly if your infrastructure is already HTTP-only.
• Pub/sub: Redis vs. Kafka. Redis pub/sub is fast and cheap to create one channel per stream for 1M streams, but it is fire-and-forget: a fan-out server that drops its subscription misses whatever was published in the meantime. That is acceptable here because we favour availability and late or reconnecting clients backfill from Cassandra. Kafka gives durable, replayable delivery, but one topic or partition per stream doesn’t scale to 1M streams and adds latency, so it fits better as the feed for moderation and analytics consumers.
• Write path: persist then publish vs. publish then persist. Writing to Cassandra first means every comment a viewer sees is in history and visible to moderation. It costs a database write inside the ~1 second budget and ties posting to Cassandra’s health. Publishing first and persisting asynchronously shaves latency, at the risk of showing a comment that never makes it into history.
• Ordering: TIMEUUID vs. a per-stream sequencer. Time-ordered IDs need no central counter, but clock skew between write servers can occasionally invert two comments. A per-stream sequence number gives strict order but funnels every comment on a viral stream through one counter, which is exactly the hot spot we are trying to avoid. Use a sequencer only when strict order matters and per-stream volume is modest, as in a 1:1 or small-group chat.
• Live delivery: sampling vs. delivering every comment. Sampling caps each client at 20–50 comments per second, whatever the stream does. The cost is that most viewers never see most comments, and the selection rules (friends, verified users, liked comments) become product decisions that need tuning and can look unfair. Below a per-stream rate threshold, deliver everything; switch to sampling only once the stream gets hot.
• Hot-stream fan-out: one channel vs. a sharded channel or relay tier. A single stream:{id} channel is simplest and enough for nearly every stream. A mega-stream overloads that one broker node, so we shard the channel or add root relays, paying with more publishes per comment and more moving parts. Turn it on per stream, by viewer count, not globally.

Key Takeaways

1. The pattern is persist → publish → fan out. Persist for history and moderation, publish once to a per-stream channel, fan out from connection servers.
2. A pub/sub channel per stream decouples writers from the hundreds of WebSocket servers and makes per-stream scaling just “add more subscribers.”
3. Fan-out cost is (comments × servers), not (comments × viewers) — the final server-to-viewer hop is a cheap in-memory loop.
4. The real limit is the human, not the network. On a viral stream, sample and aggregate the live feed; no client should receive thousands of comments per second.
5. Shard a hot stream across many fan-out servers (and shard the channel itself for mega-streams) so no single server or broker node is the bottleneck.
6. Handle the join seam — load history over REST, then subscribe live, with dedup on a time-ordered id so late joiners see a smooth timeline.