Admin Spaces inventory once missed a freshly written folder on S3 (seen once on files-bao, not reproduced) #953

Closed
opened 2026-10-17 12:51:20 +00:00 by Cordy · 0 comments
Owner

Seen during the #934 dogfood check on files-bao (Garage, v0.6.347) on 2026-10-17. The second attempt did not reproduce it.

What happened

  1. An empty folder marker spaces/Café-922/.cairn-dir (name stored decomposed) was written straight into the bucket.
  2. At 12:39:40.445 the admin inventory GET /api/v1/spaces answered 1534 bytes, with no entry for the folder.
  3. Then 68 ms later, POST /api/v1/spaces {"takeOver":true} found the folder and answered 201.

So the take-over found the folder while the inventory listing did not show it a moment earlier.

Second attempt (could not reproduce)

  • spaces/Café-934b/.cairn-dir was written the same way.
  • The inventory at 12:50:15 listed it as an orphan (1629 bytes).
  • A rig test of a decomposed folder holding only the marker, on posix and s3, also lists it as an orphan.

Possible causes to check

  • Garage listing timing. The inventory lists spaces/ with a delimiter. The take-over reads the folder's own prefix or marker. Probe P6 checked list-after-write only without a delimiter.
  • Write-to-inventory interval. The first marker may have been written seconds before the inventory call, while the second sat for minutes.
  • If it is listing lag, admins could miss a folder for a short time after an outside write. Nothing is lost, and the next reload shows it.

To do

  • Extend the Garage probe with a delimiter listing taken immediately after a PUT, repeated many times.
  • If that shows lag, document it, or have the inventory retry once.
  • If it never reproduces, close this issue with the probe result.
Seen during the #934 dogfood check on files-bao (Garage, v0.6.347) on 2026-10-17. The second attempt did not reproduce it. ## What happened 1. An empty folder marker `spaces/Café-922/.cairn-dir` (name stored decomposed) was written straight into the bucket. 2. At 12:39:40.445 the admin inventory `GET /api/v1/spaces` answered 1534 bytes, with no entry for the folder. 3. Then 68 ms later, `POST /api/v1/spaces {"takeOver":true}` found the folder and answered 201. So the take-over found the folder while the inventory listing did not show it a moment earlier. ## Second attempt (could not reproduce) - `spaces/Café-934b/.cairn-dir` was written the same way. - The inventory at 12:50:15 listed it as an orphan (1629 bytes). - A rig test of a decomposed folder holding only the marker, on posix and s3, also lists it as an orphan. ## Possible causes to check - **Garage listing timing.** The inventory lists `spaces/` with a delimiter. The take-over reads the folder's own prefix or marker. Probe P6 checked list-after-write only without a delimiter. - **Write-to-inventory interval.** The first marker may have been written seconds before the inventory call, while the second sat for minutes. - If it is listing lag, admins could miss a folder for a short time after an outside write. Nothing is lost, and the next reload shows it. ## To do - Extend the Garage probe with a delimiter listing taken immediately after a PUT, repeated many times. - If that shows lag, document it, or have the inventory retry once. - If it never reproduces, close this issue with the probe result.
Cordy closed this issue 2026-10-27 03:15:08 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#953
No description provided.