HTTPdeletesoft deleteidempotencytombstonesretention

DELETE: What Does Gone Mean?

Hard delete, soft delete, async purge — three different promises hiding behind one method. DELETE is idempotent (the retry that gets 404 still succeeded), but what deletion *means* — recoverable? invisible? eventually erased? — is a domain contract HTTP cannot write for you.

Follow the failure

Frame the contract

API design starts with a consumer, a design question and a guarantee — never with a URL.

Design question
When a client DELETEs this resource, what is actually promised — immediate erasure, hidden-but-recoverable, or a purge process — and what does a retry see?
Consumers
Clients removing things — plus compliance officers asking whether "deleted" satisfies a data-erasure request, support agents asked to undo a mistake, and integrators whose retry logic needs to know whether the second DELETE's 404 is success or a bug.
The promise
Deletion has one documented meaning per resource: what happens to the data, whether and how long recovery is possible, what subsequent reads return, and that retried deletes converge on "gone" without error noise.
RequirementConsumersResource ModelStyleContractValidationAuthorizationErrorsIdempotencyPaginationVersioningObservabilityEvolutionTrade-offs

Three deletions wearing one method

A DELETE /documents/42 that returns 204 has told the client almost nothing. Behind that response, real systems do one of three things. Hard delete: the row is gone now — cheapest, honest, unrecoverable; a mis-click or a buggy script is permanent. Soft delete: a deleted_at flag hides the resource from normal reads — recoverable, auditable, and the source of a new obligation: every query, index, uniqueness constraint and count in the system must now agree on whether deleted rows exist (the storage-side mechanics of delete-as-marker are the database domain's territory — see the tombstone model in LSM stores, where deletes are writes that shadow older data until compaction physically removes them).

Async purge is the third: DELETE marks intent, access is revoked immediately, and a background process erases data across primary storage, replicas, search indexes, caches and backups over hours or days. For any system with real data plumbing this is the only *honest* shape for full erasure — synchronous erasure across five datastores inside one HTTP request is a fiction. The contract question it raises: does the API admit the process (202 Accepted + a deletion status to poll) or pretend instantaneity (204 and hope nobody checks the search index an hour later)?

The choice is a domain decision, not a technical one. User accounts and anything under data-protection law tend to need soft-delete windows *and* eventual hard purge; financial records often must not be deletable at all, only voided (The "Everything Is CRUD" Trap again — "delete" may really be a domain state transition); ephemeral resources (sessions, drafts) hard-delete freely. Per resource, the docs must answer: recoverable? for how long? by whom? and when is the data actually gone?

The three deletion contracts
ModelAfter DELETE, a GET returnsRecoveryHonest when
Hard delete404 immediately, everywhereNone — backups at bestData is ephemeral or worthless after removal (sessions, drafts)
Soft delete404 (or 410) on normal reads; visible to admin/trash endpointsWindowed undelete, documented durationHuman mistakes are likely and reversal is a product feature
Async purge404 for access at once; GET /deletions/{id} shows purge progressUntil purge completes, maybeErasure spans indexes, caches, replicas, backups — i.e., any real system promising true erasure

The retry question: 404 is a success

DELETE is idempotent: deleting twice leaves the world in the same state as deleting once (see HTTP Methods Are Promises). But idempotent effect does not mean identical responses, and here lives a small design decision with outsized log noise: the first DELETE returns 204 No Content; the retry — the original response was lost, so the client sent it again — finds nothing there. 404? Or 204 again?

Both are defensible; the contract just has to pick and say so. Returning 404 is more informative ("it was not here when this request arrived") but punishes exactly the well-behaved retrying client — its monitoring counts an error for a successful operation, and naive client code surfaces a failure to the user whose thing *did* get deleted. Returning 204 for delete-of-absent treats DELETE as "ensure gone" — convergent, retry-friendly, and slightly lossy (a client that deletes the *wrong id* also gets a comforting 204). A useful middle: 404 but documented loudly as a success case for retries — "if you retried a DELETE and got 404, the operation succeeded." What is not defensible is the undocumented coin flip, because retry handling is precisely where clients need determinism.

The retried DELETE: same end state, different status — and the docs must bless it
Request
DELETE /documents/doc_42 HTTP/1.1
Authorization: Bearer <token>
# (retry — the 204 for the first attempt was lost in a timeout)
Response
HTTP/1.1 404 Not Found
Content-Type: application/json

{
  "error": {
    "code": "not_found",
    "message": "doc_42 does not exist.",
    "request_id": "req_01J…"
  }
}

# contract note: a 404 on DELETE retry means the resource is
# gone — treat it as success. (Or return 204 for ensure-gone
# semantics; either works, silence about it does not.)

What deletion leaves behind

Deletion has an aftermath the contract must cover. References: order ord_9 references deleted user usr_42 — does the order's API response now embed a tombstone ({"user": {"id": "usr_42", "deleted": true}}), a null, or a dangling id the client must handle 404s for? Webhook consumers hold the same problem in time-shifted form: a user.deleted event arrives *after* they cached the user (see Webhook Ordering: Assume None for why order cannot be assumed). Identity reuse: may a new resource ever take usr_42's id or unique email? Reuse breaks every external system holding the old reference; most contracts should promise ids are never reused, and say so.

Cascades are the blast-radius clause: deleting a project — what happens to its documents, memberships, webhooks? The options (refuse-while-nonempty with 409, cascade with an explicit inventory of what dies, or orphan-and-reassign) differ enormously in surprise potential; "delete refused: project has 3 documents" is friction, while a silent cascade is how a customer loses a workspace to one API call. High-consequence deletes earn protocol-level friction: soft-delete windows, a confirmation token, or modeling the deletion itself as an Resource or Action?-style process with its own resource — which also gives compliance the auditable erasure record it will eventually ask for.

  • References: define what reads of referencing resources return after deletion — tombstone stub, null, or documented dangling ids.
  • Identity: promise id non-reuse explicitly; uniqueness constraints (email) need a stated post-delete policy too.
  • Cascades: enumerate what dies with the parent, or refuse deletion of non-empty containers with a 409 naming the blockers.
  • Erasure timeline: for purge models, state when data leaves indexes, caches and backups — compliance will ask in writing.
  • Audit: keep an addressable record that deletion happened (who, when, why) even when the data itself is gone.

Key points

  • Hard delete, soft delete and async purge are three different promises; the resource docs must say which one DELETE makes, per resource.
  • True erasure across indexes, caches, replicas and backups is a process — model it as 202-plus-status rather than pretending instantaneity.
  • DELETE is idempotent by effect; pick and document the retry response (404-means-success or 204-ensure-gone) so client recovery is deterministic.
  • Deletion's aftermath is contract surface: dangling references, id non-reuse, cascade inventories and erasure timelines.
  • "Delete" in the product sometimes means a domain transition (void, archive, close) — model it as one instead of destroying the record.

Follow the failure

How the contract fails or gets misused, hop by hop — and what it costs when it completes.

  1. 1
    Team → API: ships DELETE /users/{id} as a hard row delete because it is one line of ORM.
  2. 2
    Product → API: "users can delete their account" becomes irreversible; support cannot undo mis-clicks, and refund history vanishes with the row.
  3. 3
    Ops → search: deleted users keep appearing in search results for hours — the index was never in the deletion path.
  4. 4
    Legal → team: an erasure request asks when data leaves backups; nobody knows, because deletion was a DELETE statement, not a process.
  5. 5
    Client → API: a retried DELETE gets 404, client code treats it as failure and re-queues forever; the dead-letter queue fills with successful operations.
What breaks
  • Irreversible loss from mistakes the product implicitly promised to absorb — support's most expensive ticket category.
  • Ghost data: deleted resources surviving in search indexes, caches and analytics, violating both user expectations and written policy.
  • Retry logic misreading 404-after-DELETE as failure produces alert noise at best and infinite redelivery loops at worst.

Design, observe, evolve

A contract decision is incomplete until you know how you would notice it failing and how it changes later.

Design the contract
  • • Choose the deletion model per resource (hard / soft / purge) from domain and compliance requirements, and document what a post-delete GET returns.
  • • Model true erasure as an observable process: 202, revoke access instantly, purge asynchronously, expose status; keep an audit record of the deletion itself.
  • • Document the retry contract explicitly: either 204-ensure-gone or 404-with-"retry means success" guidance.
  • • Specify aftermath: id non-reuse, reference behavior (tombstone stubs beat dangling ids), and cascade inventory or 409-refusal for containers.
Observe in production
  • • Reconciliation between primary storage and derived stores (search, caches) catches ghost data — count entities present downstream but deleted upstream.
  • • Track DELETE→404 rates: a steady low rate is healthy retries; a spike is a client bug or an id-confusion incident.
  • • Measure purge-pipeline lag (deletion requested → data gone everywhere) against the documented erasure timeline.
Evolve without breaking
  • • Hard delete can evolve to soft delete invisibly (reads already 404); soft-to-hard is a broken promise for anyone relying on recovery windows — treat it as breaking.
  • • Adding an explicit deletion-process resource (202 + status) beside a legacy 204 DELETE is additive; route new consumers there and migrate the rest (see [[api-migration]]).
  • • Retention and recovery windows are contract values: shortening them needs deprecation-grade notice; lengthening is safe.
What it costs
  • • Soft delete taxes every query in the system with deleted-row awareness, complicates uniqueness, and grows storage forever without a purge policy.
  • • Async purge is honest but heavy: a pipeline, status resources and reconciliation to operate — ephemeral data does not earn it.
  • • Refusing non-empty cascades protects users at the cost of multi-step client flows; silent cascades are convenient and catastrophic exactly once.

Misconceptions

Claim
“DELETE is not idempotent — the second call returns 404, not 204.”
Reality
Idempotency is about effects, not status codes: after one DELETE or five, the resource is equally gone. The 404 on retry is an informational difference, and the contract should tell clients to read it as success.
Claim
“A 204 means the data is gone.”
Reality
A 204 means the *request succeeded*. Whether data is erased, hidden, or queued for a purge that reaches backups next month is a separate promise — the one compliance and users actually care about — and it must be written down.
Claim
“Soft delete is safer, so make everything soft delete.”
Reality
Soft delete trades irreversibility for a system-wide obligation: every query, constraint, index and downstream consumer must now handle half-dead rows, and data-protection law may forbid keeping them at all. It is a tool for resources where recovery is a product feature — not a default.