#414: s3 driver — canonical-equivalence tolerant key addressing #417

Merged

Cordy merged 10 commits from s3-unicode into main

2026-09-03 23:28:48 +00:00

Cordy commented

2026-09-03 23:26:55 +00:00

Owner

Closes #414.

Root cause: CleanPath canonicalises every path to NFC (#183), including paths from the driver's own List — legacy NFD keys (macOS decomposed uploads pre-#183) listed fine but 404ed on Open/Delete, and S3's idempotent DELETE turned removal into a silent no-op. That is the exact life story of the /home/Mülltrennung.docx ghost.

Fix (canonical writes + equivalence-tolerant addressing):

NFC remains the canonical form for all new keys (CleanPath unchanged) — the Nextcloud/W3C school.
New storage.AltUnicodeForm (ASCII short-circuit, zero cost on the hot path) + driver-level resolve: when the canonical key is absent but its alternate form exists (exact object, dir marker, or prefix children), operations address the stored bytes — Stat/List/Open/Delete/Copy-source. Stat and List report backend-true paths, keeping roundtrips byte-faithful — the APFS school of form-insensitive lookup.
Overwrites land in place on a legacy twin; a canonical duplicate is never planted beside it. Syncthing-style migration-by-rename rejected: re-keying an object under its owner skews everything addressed by the old bytes (locks, owner manifests).
Helper lives in storage so the POSIX driver can adopt the same fallback if the exposure ever surfaces there.

Tests: in-process fake S3 backend (byte-opaque, like the real thing); all unicode built from code points, never literals. Covers: NFD file roundtrip via listing path and via canonical path, delete-really-deletes (no 204 illusion), in-place twin overwrite, canonical new writes even from NFD input, NFD directory prefix resolution.

Closes #414. **Root cause:** `CleanPath` canonicalises every path to NFC (#183), including paths from the driver's **own List** — legacy NFD keys (macOS decomposed uploads pre-#183) listed fine but 404ed on Open/Delete, and S3's idempotent DELETE turned removal into a silent no-op. That is the exact life story of the `/home/Mülltrennung.docx` ghost. **Fix (canonical writes + equivalence-tolerant addressing):** - NFC remains the canonical form for all new keys (CleanPath unchanged) — the Nextcloud/W3C school. - New `storage.AltUnicodeForm` (ASCII short-circuit, zero cost on the hot path) + driver-level `resolve`: when the canonical key is absent but its alternate form exists (exact object, dir marker, or prefix children), operations address the stored bytes — Stat/List/Open/Delete/Copy-source. Stat and List report backend-true paths, keeping roundtrips byte-faithful — the APFS school of form-insensitive lookup. - Overwrites land **in place** on a legacy twin; a canonical duplicate is never planted beside it. Syncthing-style migration-by-rename rejected: re-keying an object under its owner skews everything addressed by the old bytes (locks, owner manifests). - Helper lives in `storage` so the POSIX driver can adopt the same fallback if the exposure ever surfaces there. **Tests:** in-process fake S3 backend (byte-opaque, like the real thing); all unicode built from code points, never literals. Covers: NFD file roundtrip via listing path and via canonical path, delete-really-deletes (no 204 illusion), in-place twin overwrite, canonical new writes even from NFD input, NFD directory prefix resolution.

Cordy added 10 commits

2026-09-03 23:26:56 +00:00