A global item id index (oc:fileid, WOPI ids that survive renames) #1056

Closed
opened 2026-10-31 07:48:20 +00:00 by Cordy · 4 comments
Owner

Problem. Cairn identifies items by path. #835 added a 128-bit item id per file and folder in each folder's owner records (ownermeta, .cairn-meta.json), but only as a guard: a link or share stores its item's id and is refused when a different item is at its path. There is no way to find an item by its id. Consumers that need one:

  • WebDAV sync clients (oc:fileid, Nextcloud and ownCloud clients use it to detect moves instead of delete plus upload);
  • the WOPI host: file ids are derived from the path (office.PathFromID), so renaming a document while it is open in the editor breaks the session;
  • links and shares that should survive a rename done outside Cairn (today they answer Item not found).

Today. Decision 10 of #835: no global id index. The id lives only in the parent folder's manifest; ownermeta writes are serialised per process (residual 3 of #835 spec §13), and a move outside Cairn does not carry the manifest entry.

Proposal (to research first).

  • A state index id to backend path, maintained by the ownermeta layer on every create, move, copy and delete, record-level for #742, with a rebuild by walking the manifests.
  • Or resolve by id lazily: keep the path as the primary key and use the index only as a fallback when the path no longer holds the id.
  • Expose oc:fileid (and oc:id) in PROPFIND once the index exists; derive WOPI file ids from the item id.
  • Cost and failure modes to weigh: a second source of truth on posix and s3, an index write per mutation, garbage collection, behaviour when the index and the manifests disagree (the manifests win).

Acceptance. A decision record (research, then a design) before any code: which consumer comes first, and the measured write cost on s3.

Refs. #835 decision 10, research-835.md §6.1 option D and Q10, spec §14.

Follow-up of #835 (v0.6.376).

**Problem.** Cairn identifies items by path. #835 added a 128-bit item id per file and folder in each folder's owner records (`ownermeta`, `.cairn-meta.json`), but only as a guard: a link or share stores its item's id and is refused when a different item is at its path. There is no way to find an item by its id. Consumers that need one: - WebDAV sync clients (`oc:fileid`, Nextcloud and ownCloud clients use it to detect moves instead of delete plus upload); - the WOPI host: file ids are derived from the path (`office.PathFromID`), so renaming a document while it is open in the editor breaks the session; - links and shares that should survive a rename done outside Cairn (today they answer **Item not found**). **Today.** Decision 10 of #835: no global id index. The id lives only in the parent folder's manifest; ownermeta writes are serialised per process (residual 3 of #835 spec §13), and a move outside Cairn does not carry the manifest entry. **Proposal (to research first).** - A state index id to backend path, maintained by the ownermeta layer on every create, move, copy and delete, record-level for #742, with a rebuild by walking the manifests. - Or resolve by id lazily: keep the path as the primary key and use the index only as a fallback when the path no longer holds the id. - Expose `oc:fileid` (and `oc:id`) in PROPFIND once the index exists; derive WOPI file ids from the item id. - Cost and failure modes to weigh: a second source of truth on posix and s3, an index write per mutation, garbage collection, behaviour when the index and the manifests disagree (the manifests win). **Acceptance.** A decision record (research, then a design) before any code: which consumer comes first, and the measured write cost on s3. **Refs.** #835 decision 10, research-835.md §6.1 option D and Q10, spec §14. Follow-up of #835 (v0.6.376).
Author
Owner

Research and decision record draft (2026-10-31, code at v0.6.380).

  • Consumers differ: oc:fileid needs path-to-id (already in the folder manifest read per listing); WOPI needs id-to-path only for open documents; links surviving outside-Cairn renames can't come from a manifest-based index at all (the id travels in the parent manifest).
  • No supported client reads oc:fileid (rclone source checked; Finder/Explorer/Cyberduck have no id concept); the Nextcloud desktop client does, but needs the whole Nextcloud API.
  • WOPI is broken today beyond renames: the file id is built from the opener's view path (internal/office/token.go:94). The WOPI spec requires the id to stay the same across renames, parent renames and users; owner, share recipient and space member get three different ids for one file, which likely breaks co-editing. Needs a dogfood check.
  • Measured S3 requests through the real stack (in-memory fake): new file 38, rename 94, move 98, 25-file folder rename 104, 25-file folder copy 266. An index adds 1 write per change (N+1 per folder rename if it stores paths); Garage has no CAS, so two instances would lose index updates.
  • Side findings: folder-copy id re-mint ignores write errors (duplicate ids possible); the editor lock does not move when a parent folder is renamed.

Recommended: no global index now. Fix WOPI first: an opaque id derived from the item id, a small open-session table that follows moves made through Cairn (including parent renames), every WOPI call re-checks the item (mismatch answers 404). A future global index belongs in the #742 coordination store, not in S3 objects. Questions are being asked now. Full file: research-1056.md.

**Research and decision record draft (2026-10-31, code at v0.6.380).** - Consumers differ: `oc:fileid` needs path-to-id (already in the folder manifest read per listing); WOPI needs id-to-path only for open documents; links surviving outside-Cairn renames can't come from a manifest-based index at all (the id travels in the parent manifest). - No supported client reads `oc:fileid` (rclone source checked; Finder/Explorer/Cyberduck have no id concept); the Nextcloud desktop client does, but needs the whole Nextcloud API. - **WOPI is broken today beyond renames:** the file id is built from the opener's view path (`internal/office/token.go:94`). The WOPI spec requires the id to stay the same across renames, parent renames and users; owner, share recipient and space member get three different ids for one file, which likely breaks co-editing. Needs a dogfood check. - Measured S3 requests through the real stack (in-memory fake): new file 38, rename 94, move 98, 25-file folder rename 104, 25-file folder copy 266. An index adds 1 write per change (N+1 per folder rename if it stores paths); Garage has no CAS, so two instances would lose index updates. - Side findings: folder-copy id re-mint ignores write errors (duplicate ids possible); the editor lock does not move when a parent folder is renamed. **Recommended:** no global index now. Fix WOPI first: an opaque id derived from the item id, a small open-session table that follows moves made through Cairn (including parent renames), every WOPI call re-checks the item (mismatch answers 404). A future global index belongs in the #742 coordination store, not in S3 objects. Questions are being asked now. Full file: research-1056.md.
Author
Owner

Decision record (Nikola, 2026-10-31): recommendation approved. No global id index now; if one is needed later it belongs in the #742 coordination store, not in S3 objects. This issue becomes the WOPI batch: an opaque WOPI file id derived from the item id, an open-session table that follows moves made through Cairn (including parent folder renames), every WOPI call re-checks the item (mismatch answers 404), a document moved out of the opener's reach ends the session with a clear message and keeps the last save. Native Cairn apps are planned after go-live, so the item id is added to the change feed and as a cairn:item-id WebDAV property. No oc:fileid for third-party clients now. Links surviving renames done outside Cairn: documented limit. Side findings (folder-copy id re-mint ignores write errors; editor lock not moved on a parent rename) are fixed in the batch. Next: whole-page editor mockups, then spec and build.

**Decision record (Nikola, 2026-10-31): recommendation approved.** No global id index now; if one is needed later it belongs in the #742 coordination store, not in S3 objects. This issue becomes the WOPI batch: an opaque WOPI file id derived from the item id, an open-session table that follows moves made through Cairn (including parent folder renames), every WOPI call re-checks the item (mismatch answers 404), a document moved out of the opener's reach ends the session with a clear message and keeps the last save. Native Cairn apps are planned after go-live, so the item id is added to the change feed and as a `cairn:item-id` WebDAV property. No `oc:fileid` for third-party clients now. Links surviving renames done outside Cairn: documented limit. Side findings (folder-copy id re-mint ignores write errors; editor lock not moved on a parent rename) are fixed in the batch. Next: whole-page editor mockups, then spec and build.
Author
Owner

Mockup approved (Nikola, 2026-11-01): 1056-wopi-follow-mockup.html, 9 UI keys, and the open points as recommended: the editor page checks every 5 s with unsaved edits, 30 s otherwise and on tab focus; Collabora may show the old name until reopen (check on the dogfood); the banner does not name who moved it; display names in Collabora's user list; the lock belongs to the editor session (first opener, passed on) so every participant can save; no warning to the mover; the feed/WebDAV item id is a separate HMAC-derived id; a revoked share ends the session the same way; 503 item-id-unavailable only when the records can't be read. Next: spec and build.

**Mockup approved (Nikola, 2026-11-01):** `1056-wopi-follow-mockup.html`, 9 UI keys, and the open points as recommended: the editor page checks every 5 s with unsaved edits, 30 s otherwise and on tab focus; Collabora may show the old name until reopen (check on the dogfood); the banner does not name who moved it; display names in Collabora's user list; the lock belongs to the editor session (first opener, passed on) so every participant can save; no warning to the mover; the feed/WebDAV item id is a separate HMAC-derived id; a revoked share ends the session the same way; 503 `item-id-unavailable` only when the records can't be read. Next: spec and build.
Cordy closed this issue 2026-11-02 13:29:16 +00:00
Author
Owner

Released in v0.6.386 (PR #1069), live on both dogfoods since 15:39 UTC (pods ready, no new WARN or ERROR lines).

Editor sessions

  • The WOPI id comes from the item id (the new item-id-key), and a session table follows renames and moves, including parent-folder and space renames.
  • All participants share one session lock, released record by record. Read-only participants take no part in the lock.
  • Every call checks the item again. A session ends as moved-away, replaced or deleted, and the editor page shows an end banner with Download a copy and Open folder.
  • Calls made during a move wait up to 5 s; a missing item gets a 30 s grace before the session ends.

Item id for apps: id in the change feed, and cairn:item-id over WebDAV.

Locks (decisions after research-1056-folderlocks):

  • A folder holding another person's lock cannot be deleted, trashed or moved.
  • Live editor locks follow a move.
  • A transfer carries locks and editors along.
  • Delete unread files and space delete break locks, with break-lock in the audit and the holder's bell.
  • 423 answers name lockOwner and lockPath.

Also fixed

  • An s3 save during a folder rename was lost (write fence and keep-check).
  • An administrator's restore no longer re-encrypts the owner's records to the administrator's key.

Follow-ups filed: #1073 (lock counts in the admin dialogs), #1074 (item id on the feed's root entries).

Live checks, including the Collabora dogfood checks, are in #1005.

Released in **v0.6.386** (PR #1069), live on both dogfoods since 15:39 UTC (pods ready, no new WARN or ERROR lines). **Editor sessions** - The WOPI id comes from the item id (the new `item-id-key`), and a session table follows renames and moves, including parent-folder and space renames. - All participants share one session lock, released record by record. Read-only participants take no part in the lock. - Every call checks the item again. A session ends as `moved-away`, `replaced` or `deleted`, and the editor page shows an end banner with Download a copy and Open folder. - Calls made during a move wait up to 5 s; a missing item gets a 30 s grace before the session ends. **Item id for apps:** `id` in the change feed, and `cairn:item-id` over WebDAV. **Locks** (decisions after research-1056-folderlocks): - A folder holding another person's lock cannot be deleted, trashed or moved. - Live editor locks follow a move. - A transfer carries locks and editors along. - Delete unread files and space delete break locks, with `break-lock` in the audit and the holder's bell. - 423 answers name `lockOwner` and `lockPath`. **Also fixed** - An s3 save during a folder rename was lost (write fence and keep-check). - An administrator's restore no longer re-encrypts the owner's records to the administrator's key. **Follow-ups filed:** #1073 (lock counts in the admin dialogs), #1074 (item id on the feed's root entries). Live checks, including the Collabora dogfood checks, are in #1005.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#1056
No description provided.