Split out of #1225, where a review (P1) identified a window in which the chunkblob sweeper can unlink a payload a live PUT has already written.
The residual window
#1225 narrows this but does not close it. The sweeper now refuses to unlink when the local payload's writtenAtTS >= entry.CommitTS, which catches a PUT that re-anchors the SHA after the blob became reclaimable.
What remains is a PUT already in flight across the dereference:
- PUT writes the payload locally at
T_w
- the SHA's last existing reference is removed; RC hits zero at
T_q > T_w, queueing the blob
- the grace window elapses; the sweeper stats the payload and reads
T_w < T_q, so the new guard does not fire
- the sweeper's conditional Raft delete still sees count 0, because the PUT has not committed its chunkref
- the sweeper unlinks the payload
- the PUT commits its reference — pointing at bytes that are gone
Step 2 requires the same content to be referenced and then dereferenced while this PUT is in flight, which dedup makes reachable: two objects with identical content, one deleted.
What this needs
A writer-side pin covering the PUT's local payload write through its reference commit, which the sweeper consults before unlinking. Roughly:
- the PUT registers an in-flight marker for the SHA before writing the payload
- it clears the marker only after its chunkref/RC mutation commits
reclaim refuses to unlink while a marker exists, leaving the queue entry for a later pass
Whether the marker is node-local (the payload is node-local, so a local marker covers the node doing the unlink) or replicated is the main design question, along with how it is reclaimed if the PUT crashes between the two points — a stale pin must expire rather than block reclamation forever.
Because it changes the S3 PUT write path and adds crash-recovery semantics, this wants a *_proposed_*.md design doc before implementation, per CLAUDE.md.
Related
https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE
Split out of #1225, where a review (P1) identified a window in which the chunkblob sweeper can unlink a payload a live PUT has already written.
The residual window
#1225 narrows this but does not close it. The sweeper now refuses to unlink when the local payload's
writtenAtTS >= entry.CommitTS, which catches a PUT that re-anchors the SHA after the blob became reclaimable.What remains is a PUT already in flight across the dereference:
T_wT_q > T_w, queueing the blobT_w < T_q, so the new guard does not fireStep 2 requires the same content to be referenced and then dereferenced while this PUT is in flight, which dedup makes reachable: two objects with identical content, one deleted.
What this needs
A writer-side pin covering the PUT's local payload write through its reference commit, which the sweeper consults before unlinking. Roughly:
reclaimrefuses to unlink while a marker exists, leaving the queue entry for a later passWhether the marker is node-local (the payload is node-local, so a local marker covers the node doing the unlink) or replicated is the main design question, along with how it is reclaimed if the PUT crashes between the two points — a stale pin must expire rather than block reclamation forever.
Because it changes the S3 PUT write path and adds crash-recovery semantics, this wants a
*_proposed_*.mddesign doc before implementation, per CLAUDE.md.Related
reclaimcomment and commit message record this residual case precisely.https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE