Unicode normalization: NFC path 404s against NFD-stored filename (umlauts et al.) #183

New issue

Closed

opened 2026-08-10 19:12:12 +00:00 by Cordy · 0 comments

Cordy commented

2026-08-10 19:12:12 +00:00

Owner

Found while live-probing #175 on the dogfood: Mülltrennung.docx is stored with NFD (decomposed u + combining diaeresis — the macOS/Finder/WebDAV convention). A request for the byte-wise NFC spelling of the visually identical name — which is what gets produced by typing the name, most Linux tools, and any non-Mac client — answers 404 not found: file does not exist on every API (verified against POST /api/v1/shares; the same lookup underlies files, rename, holds, peering sends).

Consequences: "the file is right there in the listing but the API says it doesn't exist" — for any name with umlauts/accents, whenever producer and consumer normalize differently. Cross-client interop (Mac upload → Windows/Linux access, or typed paths) breaks silently, and a legal hold placed on the NFC spelling would silently not match the NFD object.

Fix direction to decide: normalize to NFC at storage.CleanPath (the single documented chokepoint) for both incoming paths and driver-returned listing names — one place, covers every surface. Care points: existing NFD-named objects on disk/bucket need a lookup fallback (try NFC, then NFD) or a one-time note that listings return the stored form; WebDAV clients (Finder!) must keep round-tripping their own names; and posix filenames are bytes, so the driver must not "correct" names on disk, only compare normalization-insensitively. Tests with an NFD fixture either way.

Found while live-probing #175 on the dogfood: `Mülltrennung.docx` is stored with **NFD** (decomposed `u` + combining diaeresis — the macOS/Finder/WebDAV convention). A request for the byte-wise **NFC** spelling of the visually identical name — which is what gets produced by typing the name, most Linux tools, and any non-Mac client — answers `404 not found: file does not exist` on every API (verified against `POST /api/v1/shares`; the same lookup underlies files, rename, holds, peering sends). Consequences: "the file is right there in the listing but the API says it doesn't exist" — for any name with umlauts/accents, whenever producer and consumer normalize differently. Cross-client interop (Mac upload → Windows/Linux access, or typed paths) breaks silently, and a legal hold placed on the NFC spelling would silently not match the NFD object. **Fix direction to decide:** normalize to NFC at `storage.CleanPath` (the single documented chokepoint) for both incoming paths *and* driver-returned listing names — one place, covers every surface. Care points: existing NFD-named objects on disk/bucket need a lookup fallback (try NFC, then NFD) or a one-time note that listings return the stored form; WebDAV clients (Finder!) must keep round-tripping their own names; and posix filenames are bytes, so the driver must not "correct" names on disk, only compare normalization-insensitively. Tests with an NFD fixture either way.