workflow automation reliability · field note

SHOPIFY-PRIVACY-WEBHOOK-01 — implement data request and redact handlers

A working diagnosis of this problem and the sequence that actually resolves it, written from the operational side rather than the marketing side.

Reported context: ## Status OPEN — current compliance defect discovered while auditing #979 on 2026-09-09. ## Defect `apps/backend/src/api/shopify/shopify.webhook.router.ts` currently routes Shopify's mandatory privacy topics: - `customers/data_request` - `customers/redact` - `shop/redact` to an immediate `200 { status: 'received' }` response only. No export, deletion, anonymization, durable request tracking, or redaction workflow was

What is actually going on

Webhook consumers that respond slowly cause upstream providers to retry, multiplying load during incidents.

Automation failures cluster around unhandled rate limits and missing idempotency keys, which produce duplicate side effects under retry.

The sequence that works

  1. Add an idempotency key to every write operation so retries cannot duplicate effects.
  2. Wrap every external call in a retry with exponential backoff and a hard attempt ceiling, and log the final state either way.
  3. Respond to webhooks within the provider timeout by acknowledging immediately and processing asynchronously.
  4. Instrument a dead-letter queue and a heartbeat. Alert on absence of expected events, not only on errors.
  5. Load test the workflow at the provider's documented rate limit to find the real breaking point before production does.

Where people lose time

Verify before you trust it

Run the checks above against your own data instead of adopting them at face value. The failure modes are specific enough that a single afternoon of measurement will either confirm this diagnosis or rule it out — both outcomes are useful, and guessing is not.

Automation Workflow Pack
A ready-to-use package for exactly this workflow — $27.55, delivered immediately.
Get Automation Workflow Pack · $27.55
Sold via Gumroad, which acts as merchant of record and handles payment and applicable taxes. All sales are final; see the return policy before purchase.