s3: Rename of a folder merges into an existing destination folder without checking #960

Closed
opened 2026-10-18 11:35:57 +00:00 by Cordy · 1 comment
Owner

Found in the #931 research (2026-10-18).

What happens: On S3, Rename of a folder (internal/storage/s3/s3.go around lines 598-668) copies the folder's keys into the destination prefix even when that prefix already holds files, so the two folders merge. On POSIX the same operation fails with ENOTEMPTY. Callers that do not check the destination first therefore behave differently per backend. The space rename's convergence is one such caller, and it can take over a folder this way (#931).

Expected: a folder rename onto an existing, non-empty destination is refused on S3 exactly as on POSIX (storage.ErrAlreadyExists or similar), unless a caller explicitly asks for a merge.

To do:

  • Add the check, without a race, to the extent S3 allows (list the destination prefix before copying, and re-check before deleting the source).
  • Audit every caller: WebDAV MOVE of folders, the REST rename, the trash, space convergence, peering.
  • Test with the stacktest rig, on posix and s3.
Found in the #931 research (2026-10-18). **What happens:** On S3, `Rename` of a folder (`internal/storage/s3/s3.go` around lines 598-668) copies the folder's keys into the destination prefix even when that prefix already holds files, so the two folders merge. On POSIX the same operation fails with ENOTEMPTY. Callers that do not check the destination first therefore behave differently per backend. The space rename's convergence is one such caller, and it can take over a folder this way (#931). **Expected:** a folder rename onto an existing, non-empty destination is refused on S3 exactly as on POSIX (`storage.ErrAlreadyExists` or similar), unless a caller explicitly asks for a merge. **To do:** - Add the check, without a race, to the extent S3 allows (list the destination prefix before copying, and re-check before deleting the source). - Audit every caller: WebDAV MOVE of folders, the REST rename, the trash, space convergence, peering. - Test with the stacktest rig, on posix and s3.
Author
Owner

Shipped in v0.6.350 (PR #970). Live check on files-bao (Garage), 2026-10-19: PASS.

I created /home/b3x-a/a.txt and /home/b3x-b/b.txt, then renamed b3x-a onto b3x-b through REST. The answer was 409 already exists, the same as POSIX. Both folders kept only their own file: b3x-b/a.txt answered 404, so nothing was merged. On v0.6.349, S3 copied a.txt into b3x-b and deleted the source.

Test folders deleted and purged.

The same refusal is the basis of #931's convergence; the two-instance S3 interleavings are covered by the tests in the PR. The cross-type case (a file onto a folder name, or a folder onto a file name) is #963.

**Shipped in v0.6.350 (PR #970). Live check on files-bao (Garage), 2026-10-19: PASS.** I created `/home/b3x-a/a.txt` and `/home/b3x-b/b.txt`, then renamed `b3x-a` onto `b3x-b` through REST. The answer was **409 `already exists`**, the same as POSIX. Both folders kept only their own file: `b3x-b/a.txt` answered 404, so nothing was merged. On v0.6.349, S3 copied `a.txt` into `b3x-b` and deleted the source. Test folders deleted and purged. The same refusal is the basis of #931's convergence; the two-instance S3 interleavings are covered by the tests in the PR. The cross-type case (a file onto a folder name, or a folder onto a file name) is #963.
Cordy closed this issue 2026-10-19 20:13:03 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#960
No description provided.