Unicode: spaces taken over from an NFD folder keep the decomposed name; membership lookup has no NFD fallback #934

Closed
opened 2026-10-15 23:39:56 +00:00 by Cordy · 5 comments
Owner

Found in the fix-923 fix round (2026-10-16).

What happens:

  • fix-923 normalises space names to NFC at the API boundary and gives the gate an NFD fallback.
  • When an admin takes over a folder stored as NFD (data from before #183, macOS clients, or tools writing directly to the bucket), the space keeps the folder's exact NFD name.
  • scope.Membership.SpaceNamed has no NFD fallback. Members may therefore not reach such a space by its NFC name.

To decide (research first):

  • Normalise the folder to NFC as part of the take-over (rename on posix; copy, then delete on S3).
  • Or add an NFD fallback to membership lookups.
  • Also check other places where NFD folder names exist under /spaces/.
Found in the fix-923 fix round (2026-10-16). **What happens:** - fix-923 normalises space names to NFC at the API boundary and gives the gate an NFD fallback. - When an admin takes over a folder stored as NFD (data from before #183, macOS clients, or tools writing directly to the bucket), the space keeps the folder's exact NFD name. - `scope.Membership.SpaceNamed` has no NFD fallback. Members may therefore not reach such a space by its NFC name. **To decide (research first):** - Normalise the folder to NFC as part of the take-over (rename on posix; copy, then delete on S3). - Or add an NFD fallback to membership lookups. - Also check other places where NFD folder names exist under `/spaces/`.
Author
Owner

Confirmed live on v0.6.346 (local live check, scratch/b7e/live346, Finding 1).

What happens: a session admin takes over the NFD-named folder Café. The space is created with the exact NFD name, but the gate and handlers normalise to NFC. So for the owner, in both forms, /state, /members, /deactivate, the files API and WebDAV all answer 404. The inventory still shows the space.

Renaming it answers 200 but takes the directory relink path:

  • On the scratch mount, the folder moved into a new directory space with no members.
  • On /tmp, an empty cafe2 folder was created and the NFD folder stayed.

An NFC folder (Crème) taken over the same way works.

Fix (next patch, v0.6.347): create the space under the NFC name, with BackendDir set to the folder's exact name. Test the take-over, rename and member access for an NFD folder on posix and s3.

**Confirmed live on v0.6.346** (local live check, `scratch/b7e/live346`, Finding 1). **What happens:** a session admin takes over the NFD-named folder `Café`. The space is created with the exact NFD name, but the gate and handlers normalise to NFC. So for the owner, in both forms, `/state`, `/members`, `/deactivate`, the files API and WebDAV all answer 404. The inventory still shows the space. **Renaming it** answers 200 but takes the directory relink path: - On the scratch mount, the folder moved into a new directory space with no members. - On `/tmp`, an empty `cafe2` folder was created and the NFD folder stayed. An NFC folder (`Crème`) taken over the same way works. **Fix (next patch, v0.6.347):** create the space under the NFC name, with `BackendDir` set to the folder's exact name. Test the take-over, rename and member access for an NFD folder on posix and s3.
Author
Owner

Decision (Nikola, 2026-10-16): this fix ships in the same release as #922 (v0.6.347). The space is created under the NFC name, and its backend folder is set to the folder's exact NFD name.

**Decision (Nikola, 2026-10-16):** this fix ships in the same release as #922 (v0.6.347). The space is created under the NFC name, and its backend folder is set to the folder's exact NFD name.
Cordy closed this issue 2026-10-17 02:10:42 +00:00
Author
Owner

Shipped in v0.6.347 (PR #952). Both dogfoods run it. I'll close this issue once the dogfood checks pass.

What changed:

  • Both drivers (posix and S3) resolve each path segment in either Unicode form, composed spelling first. ASCII paths cost nothing extra.
  • A take-over creates the space under the NFC name and keeps the folder exactly as it is stored (BackendDir). No twin folder is made.
  • Names are kept in NFC in the registry and the space store. Every backend-folder comparison goes through storage.SameName.
  • At start, an empty NFC twin left by a v0.6.346 take-over is removed. A twin that holds files is left alone with a WARN.

Proof:

  • On main, TestFault934Reproduces and TestFault934S3TwinPrefix failed: 404s after the take-over, and on S3 a new file hid the older ones. Both pass on the branch.
  • Local live check of an NFD folder planted by hand:
    • the take-over answers Café in NFC;
    • the owner and member routes return 200;
    • a member reads and uploads, and the data root shows one folder;
    • WebDAV PUT and GET work;
    • a rename leaves one folder cafe2;
    • the versions of a file inside are listed.
  • Neither dogfood logged a twin line at boot.

Follow-up for folders that exist in both spellings: #948.

**Shipped in v0.6.347** (PR #952). Both dogfoods run it. I'll close this issue once the dogfood checks pass. **What changed:** - Both drivers (posix and S3) resolve each path segment in either Unicode form, composed spelling first. ASCII paths cost nothing extra. - A take-over creates the space under the NFC name and keeps the folder exactly as it is stored (`BackendDir`). No twin folder is made. - Names are kept in NFC in the registry and the space store. Every backend-folder comparison goes through `storage.SameName`. - At start, an empty NFC twin left by a v0.6.346 take-over is removed. A twin that holds files is left alone with a WARN. **Proof:** - On main, `TestFault934Reproduces` and `TestFault934S3TwinPrefix` failed: 404s after the take-over, and on S3 a new file hid the older ones. Both pass on the branch. - Local live check of an NFD folder planted by hand: - the take-over answers `Café` in NFC; - the owner and member routes return 200; - a member reads and uploads, and the data root shows one folder; - WebDAV PUT and GET work; - a rename leaves one folder `cafe2`; - the versions of a file inside are listed. - Neither dogfood logged a twin line at boot. Follow-up for folders that exist in both spellings: #948.
Author
Owner

Dogfood checks for v0.6.347 (2026-10-17)

Scan

Neither instance has a space name or folder that is not NFC. So no earlier take-over of a decomposed folder needs repair.

files (posix on NFS): PASS

I planted a folder Café-922 by hand: bytes 43 61 66 65 cc 81 2d 39 32 32, made with a temporary debug container running as uid 1000. Then:

  • Take-over: POST /api/v1/spaces {"takeOver":true} answered 201 Café-922, NFC.
  • Owner routes: meta and members answer 200 by both spellings.
  • Saves: an upload, a second save and a read work, and the versions list holds the first save.
  • On disk: exactly one folder, the decomposed one. It holds menu.txt with link count 2, which is the hard-linked record. No NFC twin.
  • One snag, not a defect: the first upload answered 404 for about a minute. This instance runs IdP power mode, so the take-over created the identity-provider group, and live group resolution then needed its 60 s cache before Nikola's session saw it.

files-bao (Garage): not checked live yet

Planting a decomposed prefix needs the bucket's S3 credentials or the Garage CLI on TrueNAS. The S3 path is covered by TestFault934S3TwinPrefix and the stacktest s3 rows only.

I'm leaving this issue open until either files-bao is checked or Nikola decides the tests suffice.

Cleanup still pending

The take-over on files created the Keycloak groups Café-922 and Café-922-ro. Nikola is deleting them; I'll remove the folder afterwards.

## Dogfood checks for v0.6.347 (2026-10-17) ### Scan Neither instance has a space name or folder that is not NFC. So no earlier take-over of a decomposed folder needs repair. ### files (posix on NFS): PASS I planted a folder `Café-922` by hand: bytes `43 61 66 65 cc 81 2d 39 32 32`, made with a temporary debug container running as uid 1000. Then: - **Take-over:** `POST /api/v1/spaces {"takeOver":true}` answered 201 `Café-922`, NFC. - **Owner routes:** `meta` and `members` answer 200 by both spellings. - **Saves:** an upload, a second save and a read work, and the versions list holds the first save. - **On disk:** exactly one folder, the decomposed one. It holds `menu.txt` with link count 2, which is the hard-linked record. No NFC twin. - **One snag, not a defect:** the first upload answered 404 for about a minute. This instance runs IdP power mode, so the take-over created the identity-provider group, and live group resolution then needed its 60 s cache before Nikola's session saw it. ### files-bao (Garage): not checked live yet Planting a decomposed prefix needs the bucket's S3 credentials or the Garage CLI on TrueNAS. The S3 path is covered by `TestFault934S3TwinPrefix` and the stacktest s3 rows only. I'm leaving this issue open until either files-bao is checked or Nikola decides the tests suffice. ### Cleanup still pending The take-over on files created the Keycloak groups `Café-922` and `Café-922-ro`. Nikola is deleting them; I'll remove the folder afterwards.
Cordy reopened this issue 2026-10-17 09:12:44 +00:00
Author
Owner

files-bao (Garage): PASS (2026-10-17)

  • Setup. Nikola planted spaces/Café-922/.cairn-dir with a temporary Garage key. The folder bytes are 43 61 66 65 cc 81 2d 39 32 32, which is the decomposed spelling.
  • Take-over. POST /api/v1/spaces {"takeOver":true} answered 201 with the name Café-922 in NFC. Its state and members answer 200 under both spellings.
  • Writing inside the space. Two uploads of menu.txt worked. A read returns the second save, and the first save is listed as a version.
  • The bucket afterwards (listed with the key) holds exactly one prefix, spaces/Café-922/ (decomposed). It contains .cairn-dir, .cairn-meta.json, menu.txt and .cairn-versions/menu.txt/ with 2 records (the hidden current one and v1). There is no composed twin.
  • Clean-up. I deactivated and deleted the space and purged its trash entry. A later bucket list shows no Café-922 keys left.
  • Inventory listing. A second plant, Café-934b, shows in Administration › Spaces as an orphan before any take-over. The first plant did not show there for a moment; that is filed as #953 (seen once, not reproduced).

Both dogfoods now pass. Closing.

## files-bao (Garage): PASS (2026-10-17) - **Setup.** Nikola planted `spaces/Café-922/.cairn-dir` with a temporary Garage key. The folder bytes are `43 61 66 65 cc 81 2d 39 32 32`, which is the decomposed spelling. - **Take-over.** `POST /api/v1/spaces {"takeOver":true}` answered 201 with the name `Café-922` in NFC. Its state and members answer 200 under both spellings. - **Writing inside the space.** Two uploads of `menu.txt` worked. A read returns the second save, and the first save is listed as a version. - **The bucket afterwards** (listed with the key) holds exactly one prefix, `spaces/Café-922/` (decomposed). It contains `.cairn-dir`, `.cairn-meta.json`, `menu.txt` and `.cairn-versions/menu.txt/` with 2 records (the hidden current one and v1). There is no composed twin. - **Clean-up.** I deactivated and deleted the space and purged its trash entry. A later bucket list shows no `Café-922` keys left. - **Inventory listing.** A second plant, `Café-934b`, shows in Administration › Spaces as an orphan before any take-over. The first plant did not show there for a moment; that is filed as #953 (seen once, not reproduced). Both dogfoods now pass. Closing.
Cordy closed this issue 2026-10-17 12:51:26 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#934
No description provided.