Skip to content

Writer-side pin for chunkblob PUTs in flight across a dereference #1254

Description

@bootjp

Split out of #1225, where a review (P1) identified a window in which the chunkblob sweeper can unlink a payload a live PUT has already written.

The residual window

#1225 narrows this but does not close it. The sweeper now refuses to unlink when the local payload's writtenAtTS >= entry.CommitTS, which catches a PUT that re-anchors the SHA after the blob became reclaimable.

What remains is a PUT already in flight across the dereference:

  1. PUT writes the payload locally at T_w
  2. the SHA's last existing reference is removed; RC hits zero at T_q > T_w, queueing the blob
  3. the grace window elapses; the sweeper stats the payload and reads T_w < T_q, so the new guard does not fire
  4. the sweeper's conditional Raft delete still sees count 0, because the PUT has not committed its chunkref
  5. the sweeper unlinks the payload
  6. the PUT commits its reference — pointing at bytes that are gone

Step 2 requires the same content to be referenced and then dereferenced while this PUT is in flight, which dedup makes reachable: two objects with identical content, one deleted.

What this needs

A writer-side pin covering the PUT's local payload write through its reference commit, which the sweeper consults before unlinking. Roughly:

  • the PUT registers an in-flight marker for the SHA before writing the payload
  • it clears the marker only after its chunkref/RC mutation commits
  • reclaim refuses to unlink while a marker exists, leaving the queue entry for a later pass

Whether the marker is node-local (the payload is node-local, so a local marker covers the node doing the unlink) or replicated is the main design question, along with how it is reclaimed if the PUT crashes between the two points — a stale pin must expire rather than block reclamation forever.

Because it changes the S3 PUT write path and adds crash-recovery semantics, this wants a *_proposed_*.md design doc before implementation, per CLAUDE.md.

Related

https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions