Change Data Capture (CDC): Stream Every Database Change Without Touching Your App

Your orders live in Postgres. But you also need them in Elasticsearch for search, in Redis for a hot cache, in the data warehouse for analytics, and maybe pushed to another team’s service. The question every backend engineer eventually hits is: when an order changes in Postgres, how do all these other places find out? Change Data Capture is the clean answer, and once you understand it, you see it everywhere.

What CDC actually is

CDC means capturing every change — every INSERT, UPDATE and DELETE — that happens in your database, and turning it into a stream of events that other systems can consume. The elegant part is that, done properly, your application does not change at all. It keeps writing to the database exactly like before; CDC quietly watches the changes from the side.

The key realisation: your database is already recording every change, in order, for its own replication and crash recovery. Postgres has the WAL (write-ahead log); MySQL has the binlog. CDC just reads that log. You are not adding a new source of truth — you are tapping into one the database already maintains for free.

Three ways to capture changes (only one scales)

People reinvent CDC badly all the time, so it is worth seeing the three approaches side by side:

Query-based (polling). Run SELECT ... WHERE updated_at > ? every few seconds. It is simple and needs no special database access, but it is weak: it misses deletes entirely (a deleted row has no updated_at to find), it misses intermediate updates between two polls, it puts constant load on the DB, and it allways lags. Fine for a quick hack, not for serious use.
Trigger-based. Put database triggers on your tables that write change rows into an audit table. This catches everything, but now every single write carries extra trigger work on the hot path, and you have more database logic to maintain and debug. It works, but it taxes your writes.
Log-based (the real one). Read the WAL/binlog directly, usually with a tool like Debezium. This is the winner by a mile: you get every change in commit order, with almost zero load on your actual queries (you are reading a log, not running queries), it catches deletes and every update, and the application is completely untouched. The only cost is some setup and needing log access on the DB.

For anything real, log-based CDC is the right default.

The gotchas nobody mentions

CDC feels like magic, and that is exactly why teams get burnt. A few things to go in knowing:

Ordering is per-table (or per-partition), not global. Changes to a single table arrive in order, but if two different tables must stay mutually consistent, that is your problem to handle, not CDC’s.
Delivery is at-least-once. The same change can arrive twice (after a connector restart, for example). So your consumers must be idempotent — applying the same change twice should have no extra effect. This ties straight into the idempotency pattern; please do read that one also.
The initial snapshot can be heavy. Before streaming live changes, CDC usually takes a full snapshot of the existing table. For a huge table this is not light, so plan the first load carefully.
Schema changes ripple. If you drop or rename a column, every downstream consumer sees it. Version your events and coordinate schema changes — don’t just ALTER TABLE on a Friday evening and go home.
It is still eventually consistent. Consumers lag the source by a little. Same mindset as CQRS — design for the small delay.

None of these are blockers. They are simply the things that bite teh teams who assumed CDC was effortless. Know them up front, and CDC is honestly one of the best tools in the whole backend toolbox.

Where you will use it

Cache invalidation that actually works — when the row changes, the stream tells your cache to refresh. No more stale cache guesswork.
Search indexing — keep Elasticsearch in sync with the DB automatically.
Analytics / data warehouse — stream changes into the warehouse continuously instead of a nightly batch dump (see the Data Engineering course).
Microservice integration — one service’s DB changes become events other services react to. This is also the reliable publishing mechanism behind the Outbox pattern.
Zero-downtime migrations — stream from the old database to the new one while both run.

Takeaways

CDC streams every insert/update/delete out of your DB as ordered events, with the app untouched.
Prefer log-based CDC (read the WAL/binlog with Debezium) over polling (misses deletes, lags) or triggers (write overhead).
Plan for the gotchas: per-table ordering, at-least-once delivery (so consumers must be idempotent), heavy initial snapshot, and schema-change ripples.
It powers cache invalidation, search indexing, analytics pipelines, service integration and live migrations.

Part of the event-driven data patterns series: CQRS, Event Sourcing, the Outbox pattern, and idempotency. For the streaming side, the Apache Kafka and Data Engineering courses go much deeper.