Build your own webhook transport

A customer pays, your service commits the order, and a partner’s fulfillment endpoint times out. Did the partner receive the event? Will a retry create a second shipment? When support opens the delivery record, can they tell what happened without searching three systems?

The HTTP request is the least interesting part of a webhook system.

The hard parts are the state around the request. The sender must know which endpoint subscribed to the event, prove that the body is authentic, retry an ambiguous timeout without creating a second business effect, and preserve enough attempt history for support to explain the outcome. A single fetch() call provides none of those guarantees by itself.

When one service needs to notify a small set of customer or partner systems, I prefer an explicit webhook transport over adding a broker by reflex. We will build that transport from subscriptions in Postgres, outbox-backed delivery rows, signed HTTP requests, bounded retries, and an inbox on the receiving side.

This post stands alone, but it assumes you already have a durable claim loop: database rows that workers lease, process, and retry safely. If you are building that foundation, start with the Durable Work in Postgres series.

Webhooks are transport, not coordination

The durable-work loop coordinates work your service owns, using a database row to remember what is pending and which worker currently holds it. A webhook solves a different problem: it transports a versioned event to a receiver that exposes HTTP. Webhooks are a good fit when there are a small number of known receivers and you want each receiver to operate without joining your broker infrastructure.

Start with a webhook endpoint registry

The registry says who should receive which event types and how to authenticate delivery. Keep endpoint state explicit so support can pause, rotate secrets, inspect failures, and replay deliveries without editing code.

Webhook registry and delivery tables

Open in Studio

CREATE TYPE webhook_endpoint_status AS ENUM (
  'active', 'paused', 'disabled'
);

CREATE TABLE webhook_endpoints (
  id              UUID PRIMARY KEY DEFAULT uuidv7(),
  tenant_id       TEXT NOT NULL,
  url             TEXT NOT NULL,
  status          webhook_endpoint_status NOT NULL DEFAULT 'active',
  secret_ref      TEXT NOT NULL,        -- points to KMS/secret store
  max_attempts    INT NOT NULL DEFAULT 12,
  created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  updated_at      TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_webhook_endpoints_active
  ON webhook_endpoints (tenant_id)
  WHERE status = 'active';

CREATE TABLE webhook_subscriptions (
  endpoint_id     UUID NOT NULL REFERENCES webhook_endpoints(id),
  event_type      TEXT NOT NULL,
  PRIMARY KEY (endpoint_id, event_type)
);

CREATE INDEX idx_webhook_subscriptions_event_type
  ON webhook_subscriptions (event_type, endpoint_id);

Create delivery rows from outbox events

The domain transaction writes one outbox row to say the event still needs publication. A relay worker reads that row, finds every active subscription for its event type, and creates one delivery row per endpoint. This fan-out is important because each receiver can succeed, fail, pause, and retry independently. HTTP workers claim the delivery rows rather than the original outbox row, so a slow partner does not hide the successful delivery state of every other subscriber.

CREATE TYPE webhook_delivery_status AS ENUM (
  'pending', 'processing', 'delivered', 'dead_letter'
);

CREATE TABLE webhook_deliveries (
  id                UUID PRIMARY KEY DEFAULT uuidv7(),
  endpoint_id       UUID NOT NULL REFERENCES webhook_endpoints(id),
  tenant_id         TEXT NOT NULL,
  event_id          TEXT NOT NULL,
  event_type        TEXT NOT NULL,
  payload           JSONB NOT NULL,
  status            webhook_delivery_status NOT NULL DEFAULT 'pending',
  attempts          INT NOT NULL DEFAULT 0,
  max_attempts      INT NOT NULL DEFAULT 12,
  claimed_by        TEXT REFERENCES workers(id),
  claimed_at        TIMESTAMPTZ,
  lease_expires_at  TIMESTAMPTZ,
  lease_generation  BIGINT NOT NULL DEFAULT 0,
  available_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  last_status_code  INT,
  last_error        TEXT,
  delivered_at      TIMESTAMPTZ,
  created_at        TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE (endpoint_id, event_id)
);

CREATE INDEX idx_webhook_deliveries_claim
  ON webhook_deliveries (available_at, created_at, id)
  WHERE status = 'pending';

CREATE TABLE webhook_delivery_attempts (
  id                UUID PRIMARY KEY DEFAULT uuidv7(),
  delivery_id       UUID NOT NULL REFERENCES webhook_deliveries(id),
  attempted_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  status_code       INT,
  error             TEXT,
  duration_ms       INT
);

INSERT INTO webhook_deliveries (endpoint_id, tenant_id, event_id, event_type, payload, max_attempts)
SELECT e.id, e.tenant_id, $event_id, $event_type, $payload, e.max_attempts
FROM webhook_endpoints e
JOIN webhook_subscriptions s ON s.endpoint_id = e.id
WHERE e.tenant_id = $tenant_id
  AND e.status = 'active'
  AND s.event_type = $event_type
ON CONFLICT (endpoint_id, event_id) DO NOTHING;

Deliver webhooks with the same claim loop

The delivery worker is another durable worker. It claims pending delivery rows, signs the request, sends HTTP, records success, or schedules retry with backoff. Each claim increments a generation number. Completion and retry updates must match that generation and the worker id, so a stale worker cannot overwrite a newer claim.

WITH picked AS (
  SELECT id
  FROM webhook_deliveries
  WHERE status = 'pending'
    AND available_at <= now()
  ORDER BY created_at, id
  LIMIT 25
  FOR UPDATE SKIP LOCKED
)
UPDATE webhook_deliveries d
SET
  status = 'processing',
  claimed_by = $worker_id,
  claimed_at = now(),
  lease_expires_at = now() + interval '90 seconds',
  lease_generation = d.lease_generation + 1,
  attempts = attempts + 1
FROM picked
WHERE d.id = picked.id
RETURNING d.*;

async function deliverWebhook(delivery, endpoint, resolveSecret) {
  const timestamp = Math.floor(Date.now() / 1000);
  const body = JSON.stringify({
    id: delivery.event_id,
    type: delivery.event_type,
    created_at: delivery.created_at,
    data: delivery.payload,
  });
  const secret = await resolveSecret(endpoint.secret_ref);
  const signature = await signWebhook(secret, `${timestamp}.${body}`);

  const res = await fetch(endpoint.url, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Webhook-Id": delivery.event_id,
      "Webhook-Timestamp": String(timestamp),
      "Webhook-Signature": signature,
    },
    body,
    redirect: "error",
    signal: AbortSignal.timeout(10_000),
  });

  if (res.status >= 200 && res.status < 300) {
    await db.markWebhookDelivered(delivery.id, res.status);
    return;
  }
  throw new RetryableDeliveryError(res.status);
}

async function signWebhook(secret, message) {
  const encoder = new TextEncoder();
  const key = await crypto.subtle.importKey(
    "raw",
    encoder.encode(secret),
    { name: "HMAC", hash: "SHA-256" },
    false,
    ["sign"],
  );
  const bytes = new Uint8Array(await crypto.subtle.sign("HMAC", key, encoder.encode(message)));
  return `v1=${Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join("")}`;
}

The timeout must stay comfortably below the lease duration. Reject redirects so a registered URL cannot silently send a signed payload somewhere you did not approve. The downloadable TypeScript includes the worker and generation checks on every state transition. The shorter excerpt above focuses on signing and sending.

Retry like a transport, not like a request handler

Webhook delivery is at-least-once. A timeout might mean the receiver processed the event but your worker never saw the response. Retries are normal, so every webhook event needs a stable id and every receiver needs dedupe.

Do not collapse that table into “retry every non-2xx.” A retry storm will not repair a revoked credential or deleted endpoint. Record the attempt, pause or dead-letter according to policy, and make the repair action visible to support.

UPDATE webhook_deliveries
SET
  status = CASE
    WHEN attempts >= max_attempts THEN 'dead_letter'::webhook_delivery_status
    ELSE 'pending'::webhook_delivery_status
  END,
  claimed_by = NULL,
  lease_expires_at = NULL,
  available_at = now() + (LEAST(POWER(2, attempts)::int * 30, 86400) * interval '1 second'),
  last_status_code = $status_code,
  last_error = $error
WHERE id = $delivery_id;

This abbreviated query shows the retry calculation. The real update must also require the expected claimed_by, lease_generation, and processing status. The downloadable implementation includes those guards.

Inbound webhooks become inbox rows

The receiver side should be boring: verify the signature, insert an inbox row with a unique idempotency key, and return 2xx after the row is durable. Processing happens later in the claim loop.

The next snippet is framework-neutral pseudocode. Use your framework’s raw-body API and validate the parsed event at the boundary.

async function receiveWebhook(req, res) {
  const rawBody = await readRawBody(req);
  await verifyWebhookSignature({
    body: rawBody,
    timestamp: req.headers["webhook-timestamp"],
    signature: req.headers["webhook-signature"],
    secret: WEBHOOK_SECRET,
  });

  const event = JSON.parse(rawBody);
  await db.query(
    `
    INSERT INTO inbox (partition_key, partition_bucket, payload, idempotency_key)
    VALUES ($1, $2, $3, $4)
    ON CONFLICT (idempotency_key) WHERE idempotency_key IS NOT NULL DO NOTHING
  `,
    [event.data.account_id, bucketFor(event.data.account_id), event, event.id],
  );

  res.status(204).send();
}

Reject signatures outside a short timestamp window as well. The event id prevents duplicate processing. The timestamp check prevents an old signed request from being replayed forever.

Make delivery inspectable and replayable

A webhook transport is only credible if support can explain a delivery without reading logs from five services.

Where a webhook registry fits

Strong fit

• Customer or partner notifications over HTTP
• Moderate fan-out where each receiver owns an endpoint
• Events already exist in your outbox
• Receivers can dedupe by event id
• You want SQL-visible delivery state and replay

Use a broker or stream when

• You need many independent internal consumers on the same log
• Replay and retention are central platform requirements
• Consumers need pull-based backpressure and consumer groups
• Delivery volume is high enough that HTTP fan-out is the bottleneck
• Ordering, partitioning, and retention need shared infrastructure

Sharp edges before you ship

• Signing: sign timestamp plus raw body. Reject old timestamps to limit replay attacks.
• Untrusted URLs: validate scheme and port at registration, block private/link-local addresses, re-check DNS at delivery, reject redirects, and enforce egress policy. Otherwise the registry is an SSRF endpoint with a friendly admin screen.
• Idempotency: every event needs a stable id, and receivers must deduplicate it.
• Secrets: support secret rotation with overlapping active secrets.
• Retries: timeouts are ambiguous. Retrying can duplicate delivery.
• Replay: replay should create a new delivery attempt, not mutate history beyond recognition.

What you should have now

The finished transport makes responsibility visible. The outbox proves that the producer intended to publish. A delivery row records what each endpoint is owed. Attempt history explains failures. HMAC signatures authenticate the body, bounded retries handle transient outages, and the receiver’s inbox makes duplicate delivery safe.

Before calling it production-ready, run three failure tests: terminate a worker during the HTTP request, return a permanent 401, and send the same event twice. The system should recover the ambiguous timeout, stop retrying the invalid credential according to policy, and apply the receiver’s business effect once. If support cannot explain all three from stored state, the transport is not finished.

Source

These companion files collect the SQL and TypeScript used in this article. The link labels distinguish full implementations from focused examples and worker skeletons.

• Webhook schema
• Maintained TypeScript implementation
• Durable work schema (workers + inbox)
• Shared TypeScript helper