Worked example / reliability

Make a webhook safe to retry.

A handler can pass its happy-path test and still update an account twice. Follow the failure window, then build a review checklist for retries and concurrent deliveries.

Start with the promise to the user

Suppose a verified event grants 10 credits to an account. Delivering that same event twice should still add only 10 credits. A different valid event should add its own credits. That is the behavior we want to preserve.

For this exercise, the payload and signature are already validated, and both the event marker and account live in one transactional database. We are deduplicating a stable event identity, not claiming that every business action has exactly one event.

Real providers can retry deliveries. Stripe’s webhook documentation, for example, describes duplicate-event handling. Always check the delivery and identity rules of your own provider.

Find the gap between the two writes

A tempting implementation checks whether an event was processed, increments the credits, and then saves an event marker. Imagine the process stops after the increment but before the marker. On the next delivery, the lookup finds nothing, so the credits are added again.

Reversing the writes creates the opposite failure: save the marker, crash before adding credits, and future retries skip work that never happened. The right question is which state changes must succeed together.

A transaction and a unique claim solve different problems

The transaction groups the marker and credit change so a failure cannot leave just one committed. A unique constraint on the event key prevents two workers from independently claiming the same event. A transaction around a plain “check, then write” is not by itself a substitute for uniqueness.

In the sketch below, processed_events has a non-null unique key on (provider, account_id, event_id). Its non-null account reference is a foreign key to accounts.id. The account must exist and remain available for the update; the application must roll back if a newly claimed event cannot update its account. All inputs are validated, and credits are represented in the database’s appropriate numeric type.

Illustrative PostgreSQL transaction · parameterized values
BEGIN;

WITH claimed AS (
  INSERT INTO processed_events (provider, account_id, event_id)
  VALUES ($1, $2, $3)
  ON CONFLICT (provider, account_id, event_id) DO NOTHING
  RETURNING event_id
)
UPDATE accounts
SET credits = credits + $4
WHERE id = $2 AND EXISTS (SELECT 1 FROM claimed);

COMMIT;

Only a newly inserted claim produces a row in claimed, so a duplicate does not increment credits. A database error must roll back the whole transaction. A transient failure should remain retryable; do not acknowledge success before the relevant work is durable.

See PostgreSQL’s unique constraints and transaction documentation for the underlying guarantees. This sketch teaches a review pattern, not a complete webhook endpoint.

Test the promise, not just the response code

Regression checklist for the credit update
ScenarioCommitted state to assert
First deliveryOne marker and one credit increment.
Sequential duplicateNo additional credit increment.
Concurrent duplicateTwo workers compete; only one increment commits.
Failure before commitNeither write persists; a later retry can succeed once.
A different eventA new increment is applied, even for the same account.

For the concurrency case, coordinate workers so their requests really overlap; two ordinary sequential calls do not exercise the race. Inspect database state after both attempts finish. Also test what happens if a commit succeeds but the response never reaches the sender.

Know where this example stops

A database transaction cannot roll back an email already sent or an external API call already accepted. Those effects need their own durable delivery and deduplication design, often involving an outbox and a retrying worker. Distinct events about the same business action may also require a business-level invariant beyond event-ID deduplication.

Before production, review signature verification, event schemas, permissions, event ordering, retry policy, retention of deduplication records, and monitoring. Deleting a marker too soon can make an old replay look new.

Now make the review decisions yourself.

Use the failure-window, concurrency, and regression-test questions on a short proposed change. You get feedback after each decision.

Try the free code review

Back to the practice guides