The Transactional Outbox Pattern: Never Lose an Event Again

This is one of those patterns that looks like nothing until you get burnt once, and then you never forget it. Picture a very ordinary service method: a user places an order, so you save the order to your database and publish an OrderPlaced event so the warehouse, the email service and everyone else can react. Two lines of code, looks totally innocent. And it is quietly broken.

The dual-write problem

The problem is that you are writing to two different systems — your database and your message broker — and there is no shared transaction across them. So whatever order you do it in, a crash in the middle leaves you inconsistent.

Walk through it:

• DB first, then publish: you commit the order to the database, and then — right before publishing — the process crashes, the broker is down, or the network blips. The order exists, but the event is lost. The warehouse never hears about it. The customer’s order just sits their, silently.
• Publish first, then DB: you publish the event, and then the database commit fails and rolls back. Now there is an event for an order that does not exist — a “ghost” event. Downstream services act on something that isn’t real.

And “just retry” does not save you — if the publish succeeded but you did not get the acknowledgement, a retry will double-publish. The hard truth is simple: you cannot make two separate systems commit atomically without a shared transaction. So we stop trying to.

The fix: write the event into the same database

Here is the neat trick. Instead of publishing to the broker inside your business transaction, you write the event as a row into an outbox table in the very same database, inside the same transaction as your business data.

Now there is only one transaction, in one system, so the usual database guarantee applies: the order row and the outbox row either both commit, or neither does. No more half-states. The event is now durably saved right next to the data it describes.

# ONE local transaction — both rows commit together, or neither does
with db.transaction():
    db.insert("orders", order)
    db.insert("outbox", {
        "id": uuid4(),
        "topic": "orders",
        "payload": OrderPlaced(order.id, order.user_id, order.total),
        "published": False,
    })
# no broker call here at all — that happens later, separately

Then a separate relay does the publishing, out of band:

# a background relay — runs continuously
def relay():
    rows = db.query("SELECT * FROM outbox WHERE published = false ORDER BY created_at")
    for row in rows:
        broker.publish(row.topic, row.payload)     # may be retried
        db.update("outbox", row.id, published=True) # mark done after the ack

The relay can be a simple poller like above, or — much nicer — you point CDC at the outbox table and let it stream new rows to the broker automatically. Either way, the business transaction is clean and the publishing is reliable.

The one rule you must remember: idempotent consumers

There is no free lunch, so here is the catch. If the relay publishes an event and then crashes before it marks the row as published, it will publish that same event again on the next run. This is a deliberate choice — the pattern gives you at-least-once delivery, which means an event is never lost, but it may be delivered more than once.

So the rule is non-negotiable: your consumers must be idempotent. Processing the same event twice must have exactly the same effect as processing it once — usually by tracking the event’s id and skipping one you have already seen. The idempotency pattern article covers exactly how to do this; the two patterns are best freinds and almost always appear together.

Think of it like this: Outbox makes sure the event is never lost. Idempotency makes sure the duplicate never hurts. Together they give you reliable event delivery, which is the whole game in event-driven systems.

When to use it

• Any time a single action must both change your data and reliably tell the outside world — place order + notify, sign up + send welcome email, payment captured + update ledger.
• Microservices that communicate through events and cannot afford to silently drop one.
• As the reliable backbone under a Saga, where each step must dependably trigger the next.

If you only ever write to your database and never publish events, you don’t need it. But the moment you see that “save, then publish” pair in your code, remember — that innocent-looking line is the dual-write trap, and the outbox is how you escape it.

Takeaways

• The dual-write problem: DB and broker are separate systems with no shared transaction, so a crash between them loses events or creates ghost events. Retries can double-publish.
• The Transactional Outbox: write the business row and an outbox row in the same DB transaction — atomic, no half-states.
• A relay (a poller, or CDC on the outbox table) publishes the events out of band and marks them done after the broker acks.
• Delivery is at-least-once, so consumers must be idempotent — Outbox stops loss, idempotency stops duplicate harm.

Part of the event-driven data patterns series: CQRS, Event Sourcing, Change Data Capture, and its essential partner idempotency. For reliable messaging in general, the Apache Kafka course is a good next stop.