Inbox Pattern (Idempotent Consumer)

Guarantee idempotent message processing in a consumer by recording each message's unique identifier in a local "inbox" table, in the same transaction as the message's side effect, before that side effect runs. A redelivered message finds its ID already recorded and is skipped, so the side effect happens exactly once even though the broker may deliver the message more than once.

At-least-once message delivery means a consumer will occasionally receive the same message more than once. A consumer that applies effects on every delivery (double-write, double-charge) produces incorrect state. Idempotency cannot be reliably implemented at the broker; it has to be enforced at the consumer.

Message brokers (Kafka, RabbitMQ, SQS) guarantee at-least-once delivery: a message can be redelivered if a consumer crashes after processing but before acknowledging, or if a redelivery timeout fires before the ack arrives. For a consumer with side effects (a database write, a charge, an email), processing the same message twice can corrupt state.

The mechanism: before processing a message, the consumer attempts to insert the message's unique ID into a local inbox table with a UNIQUE constraint on that ID, in the same database transaction as the business logic itself:

BEGIN;

INSERT INTO message_inbox (message_id) VALUES ($id)

ON CONFLICT DO NOTHING RETURNING message_id;

-- if a row came back, this message has not been seen: apply the business logic

-- if no row came back, this message was already processed: skip it

COMMIT;

-- acknowledge to the broker after the transaction commits

The safety here comes specifically from the UNIQUE constraint's atomicity, not from any check-then-act logic in application code. If two deliveries of the same message race each other, on a rebalance or a redelivery landing concurrently with the original still in flight, both attempt the INSERT, but the database guarantees only one commits the row; the other's ON CONFLICT DO NOTHING clause returns no row, and that delivery skips the business logic. A check-then-insert done as two separate steps would have a race window between them; combining the check and the write into one constrained INSERT removes that window entirely.

The guarantee this delivers is precisely scoped: it makes the business logic run exactly once for effects captured within the same local transaction as the inbox insert. It does not extend to a side effect performed outside that transaction, a direct external API call (a card charge, an email send) made from the handler after the transaction commits, since a crash between commit and that external call, or a retry of the external call itself, is not covered by the inbox's atomicity. Effects that must be exactly-once end to end need to either happen inside the same local transaction (write an outbox row recording the intent, and let a separate relay perform the external call, itself idempotent) or carry their own idempotency key that the external system honors.

The inbox table accumulates rows and needs pruning. A cleanup job removes entries older than the broker's maximum redelivery window (commonly 7 days for Kafka, configurable); an ID that cannot possibly be redelivered after that window is safe to delete.

Inbox is the consumer-side half of a producer/consumer pair with the outbox pattern, and the two are not interchangeable. Outbox solves the producer's problem: reliably getting a message published at least once without a dual-write gap between the database and the broker. Inbox solves the consumer's problem: making redelivery safe once the message arrives. Combining both gives effectively-once processing end to end, the same term outbox_pattern uses for its own guarantee, since the delivery itself remains at-least-once (broker-guaranteed) and only the local processing step is exactly-once; claiming a stronger, unqualified "exactly-once" for the combination would overclaim past what either piece actually provides.