Decision: Nextcloud "Trusted servers" compatibility (status.php + federation handshake + directory sync?) #455

Closed
opened 2026-09-09 20:09:20 +00:00 by Cordy · 5 comments
Owner

Context

Nikola tried adding files-bao to NC's Federation admin → "No server to federate with found". NC 34 probed /.well-known/ocm/status.php → our 401 → NC log: "No Nextcloud server." Question raised: shouldn't Cairn be addable there, like the other players?

What "Trusted servers" actually is (research, 2026-09)

Not OCM. It is NC-family compat with three layers: (1) status.php probe ("is this a Nextcloud-like?"), (2) shared-secret handshake via /ocs/v2.php/apps/federation/…, (3) periodic system address book sync (CardDAV) exchanging the full user list. Its only function is share-dialog autocomplete of remote users; it gates no sharing whatsoever (confirmed in #117 research and by NC docs).

Who can be added today: ownCloud 10 yes (ships status.php, same lineage). OpenCloud/oCIS yes at probe level (ships a status.php compat endpoint). Seafile no — no status.php, cannot be added, and has no equivalent feature of its own.

Options for Cairn

  • Tier 0 — status quo: not addable; direct-address + invite sharing works regardless. Cost: NC admins get a confusing red herring in their UI.
  • Tier 1 — status.php shim only: NC accepts the add, but the entry sits forever unverified (handshake endpoints missing) — a permanently yellow/red row is arguably worse than a clean refusal.
  • Tier 2 — full compat: shim + shared-secret handshake + system address book. Green row, autocomplete works. Privacy cost: exports the complete user directory to every trusted peer — the mini-lookup-server pattern #117 explicitly rejected, and it contradicts our user-owned-contacts model.
  • Tier 3 — middle path (candidate): shim + handshake (green row) + an address book that serves only users who opted in (per-user toggle, default off). NC admins see a first-class citizen; the directory stays sovereign. Differentiator-compatible.

Ask

Decide tier. If >0: mockup for the opt-in surface first, then house TDD build; document in handbook federation-ocm.md ("appearing in Nextcloud's Trusted servers" section) either way — including, for tier 0, an explicit "why not" paragraph.

House rules: karpathy-guidelines, Superpowers brainstorming before any build.

## Context Nikola tried adding files-bao to NC's Federation admin → "No server to federate with found". NC 34 probed `/.well-known/ocm/status.php` → our 401 → NC log: "No Nextcloud server." Question raised: shouldn't Cairn be addable there, like the other players? ## What "Trusted servers" actually is (research, 2026-09) Not OCM. It is NC-family compat with three layers: (1) `status.php` probe ("is this a Nextcloud-like?"), (2) shared-secret handshake via `/ocs/v2.php/apps/federation/…`, (3) periodic **system address book sync** (CardDAV) exchanging the full user list. Its only function is share-dialog **autocomplete** of remote users; it gates no sharing whatsoever (confirmed in #117 research and by NC docs). Who can be added today: **ownCloud 10** yes (ships status.php, same lineage). **OpenCloud/oCIS** yes at probe level (ships a status.php compat endpoint). **Seafile** no — no status.php, cannot be added, and has no equivalent feature of its own. ## Options for Cairn - **Tier 0 — status quo:** not addable; direct-address + invite sharing works regardless. Cost: NC admins get a confusing red herring in their UI. - **Tier 1 — status.php shim only:** NC accepts the add, but the entry sits forever unverified (handshake endpoints missing) — a permanently yellow/red row is arguably worse than a clean refusal. - **Tier 2 — full compat:** shim + shared-secret handshake + system address book. Green row, autocomplete works. **Privacy cost: exports the complete user directory to every trusted peer** — the mini-lookup-server pattern #117 explicitly rejected, and it contradicts our user-owned-contacts model. - **Tier 3 — middle path (candidate):** shim + handshake (green row) + an address book that serves **only users who opted in** (per-user toggle, default off). NC admins see a first-class citizen; the directory stays sovereign. Differentiator-compatible. ## Ask Decide tier. If >0: mockup for the opt-in surface first, then house TDD build; document in handbook federation-ocm.md ("appearing in Nextcloud's Trusted servers" section) either way — including, for tier 0, an explicit "why not" paragraph. House rules: karpathy-guidelines, Superpowers brainstorming before any build.
Author
Owner

Tier-3 steering design (mockup ocm-directory-optin-mockup.html, rev 1, awaiting Nikola):

Four stacked gates, each owned by the right party. (1) Server on the allow-list — admin, exists today. (2) Directory compat enabled — admin, new master toggle in a Directory tab beside Status/Servers in the OCM topic (status endpoint + handshake live only while on; uses the same Apply/Discard bar). (3) Handshake completed — automatic, shown per server as verified/pending with last-sync stamp. (4) User opted in — a "Let trusted servers list me" card at the top of Federated → Contacts, default off for everyone, IdP and local users identical.

Key properties: the admin sees a live tally (e.g. "3 of 14 users opted in") but has no control to list anyone — consent stays with the user, mirroring the #117 user-owned-contacts decision. The user card names exactly what is exposed (display name + federated address, nothing else) and shows gate chips ("directory not enabled" / "2 servers verified" / "you: listed") so a toggle never silently does nothing. Opt-out removes the user from the next sync; opted-out users never appear in any address-book response. Removing a server from the allow-list drops its handshake.

Tier-3 steering design (mockup `ocm-directory-optin-mockup.html`, rev 1, awaiting Nikola): **Four stacked gates, each owned by the right party.** (1) Server on the allow-list — admin, exists today. (2) Directory compat enabled — admin, new master toggle in a **Directory** tab beside Status/Servers in the OCM topic (status endpoint + handshake live only while on; uses the same Apply/Discard bar). (3) Handshake completed — automatic, shown per server as verified/pending with last-sync stamp. (4) **User opted in** — a "Let trusted servers list me" card at the top of Federated → Contacts, default off for everyone, IdP and local users identical. Key properties: the admin sees a live tally (e.g. "3 of 14 users opted in") but has no control to list anyone — consent stays with the user, mirroring the #117 user-owned-contacts decision. The user card names exactly what is exposed (display name + federated address, nothing else) and shows gate chips ("directory not enabled" / "2 servers verified" / "you: listed") so a toggle never silently does nothing. Opt-out removes the user from the next sync; opted-out users never appear in any address-book response. Removing a server from the allow-list drops its handshake.
Author
Owner

Decisions (Nikola, 2026-09-09) — supersedes the rev-1 mockup's steering:

  1. Naming: the toggle title must cover both families. Note: "OCM peers' trusted servers" would mislabel — trusted servers is not OCM, it is Nextcloud/ownCloud-family compat. Working copy: "Appear in Nextcloud & ownCloud trusted-server lists"; final wording at mockup rev 2.
  2. No separate master toggle. Allow-listing a server is trusting it — a second "enable directory" switch is redundant and invites operator confusion. Directory compat (status endpoint + handshake) is active from the moment a legacy-dialect (NC/oC) server is added to the allow-list; removing the server drops its handshake. Pure-OCM peers (OpenCloud invites) are unaffected — no trusted-servers concept there. The Directory tab becomes visibility only: handshake state per server, last sync, entries served.
  3. Listing model: operator + quiet per-user opt-out. Trusting a server lists everyone by default (operator's conscious decision, zero user action, no confusion); a single quiet per-user escape ("don't list me to trusted servers") is honoured at sync time — kept deliberately unobtrusive, placement decided at mockup rev 2. Rationale: matches the NC/oC model UX-wise while strengthening the data-protection position, since the directory discloses every user's name + federated address to third-party servers.
  4. Kept distinct: directory listing ≠ acceptance policy. The per-server "Accept shares from" (invited contacts / any) continues to govern whose shares are accepted; being discoverable never bypasses it.

Build remains gated on: #454 root cause first, then mockup rev 2 → approval → house TDD.

**Decisions (Nikola, 2026-09-09)** — supersedes the rev-1 mockup's steering: 1. **Naming:** the toggle title must cover both families. Note: "OCM peers' trusted servers" would mislabel — trusted servers is not OCM, it is Nextcloud/ownCloud-family compat. Working copy: "Appear in Nextcloud & ownCloud trusted-server lists"; final wording at mockup rev 2. 2. **No separate master toggle.** Allow-listing a server *is* trusting it — a second "enable directory" switch is redundant and invites operator confusion. Directory compat (status endpoint + handshake) is active from the moment a legacy-dialect (NC/oC) server is added to the allow-list; removing the server drops its handshake. Pure-OCM peers (OpenCloud invites) are unaffected — no trusted-servers concept there. The **Directory tab becomes visibility only**: handshake state per server, last sync, entries served. 3. **Listing model: operator + quiet per-user opt-out.** Trusting a server lists everyone by default (operator's conscious decision, zero user action, no confusion); a single quiet per-user escape ("don't list me to trusted servers") is honoured at sync time — kept deliberately unobtrusive, placement decided at mockup rev 2. Rationale: matches the NC/oC model UX-wise while strengthening the data-protection position, since the directory discloses every user's name + federated address to third-party servers. 4. **Kept distinct:** directory listing ≠ acceptance policy. The per-server "Accept shares from" (invited contacts / any) continues to govern whose shares are accepted; being discoverable never bypasses it. Build remains gated on: #454 root cause first, then mockup rev 2 → approval → house TDD.
Author
Owner

Listing model revised (Nikola, 2026-09-09) — supersedes point 3 of the previous decision comment.

Relationship-derived, per-server listing. A user is included in the directory served to trusted server X iff:

  • they have an invited contact on X, or
  • they have sent a federated share to X or accepted one from X (accepted, not merely received — a stranger cannot force-list anyone), or
  • they explicitly enabled "list me on this server" in Federated → Contacts (per-server override; the same place offers "stop listing me" to sever a derived listing).

Default with no relationship: not listed. This is per-(user, server) — technically clean because each trusted server fetches the address book itself, authenticated by its own handshake secret, so we serve each server its own subset.

Recorded properties:

  • Granularity ends at the server wall: once listed to X, all users of X see the entry (NC-side limitation, no design of ours reaches past it). Documented in the handbook.
  • First contact still requires a typed full address (inherent chicken-and-egg); after a user's first exchange with anyone on X, everyone on X can autocomplete them.
  • Confirmed: the surviving admin surface is the allow-list alone (coarse gate), mirroring the Reva/oCIS provider-list + user-owned-contacts architecture — which is what #117 already built. Directory tab stays visibility-only.
  • Unchanged: directory listing ≠ acceptance policy ("Accept shares from" still governs).

Next step when #455 is picked up (after #454 root cause): mockup rev 2 on this model → approval → house TDD build.

**Listing model revised (Nikola, 2026-09-09) — supersedes point 3 of the previous decision comment.** **Relationship-derived, per-server listing.** A user is included in the directory served to trusted server X iff: - they have an invited contact on X, **or** - they have **sent** a federated share to X or **accepted** one from X (accepted, not merely received — a stranger cannot force-list anyone), **or** - they explicitly enabled "list me on this server" in Federated → Contacts (per-server override; the same place offers "stop listing me" to sever a derived listing). Default with no relationship: **not listed**. This is per-(user, server) — technically clean because each trusted server fetches the address book itself, authenticated by its own handshake secret, so we serve each server its own subset. Recorded properties: - Granularity ends at the server wall: once listed to X, all users of X see the entry (NC-side limitation, no design of ours reaches past it). Documented in the handbook. - First contact still requires a typed full address (inherent chicken-and-egg); after a user's first exchange with anyone on X, everyone on X can autocomplete them. - Confirmed: the surviving admin surface is the allow-list alone (coarse gate), mirroring the Reva/oCIS provider-list + user-owned-contacts architecture — which is what #117 already built. Directory tab stays visibility-only. - Unchanged: directory listing ≠ acceptance policy ("Accept shares from" still governs). Next step when #455 is picked up (after #454 root cause): mockup rev 2 on this model → approval → house TDD build.
Cordy closed this issue 2026-09-10 23:25:50 +00:00
Author
Owner

Shipped in v0.6.158, live on both dogfoods — cairn_build_info{version="v0.6.158"} verified, and https://files-bao.c0rdyceps.ch/status.php already answers the probe JSON (active because nextcloud.c0rdyceps.ch sits on the allow-list as a legacy-dialect server).

Built exactly to the locked decisions + approved rev-2 mockups, house TDD (backend red witnessed on the runner; wire formats pinned against the actual Nextcloud federation-app source):

  • Compat surface (internal/ocm/directory.go, mounted on the public mux, active only while a legacy server is allow-listed): status.php, OCS request-shared-secret (allow-list gated; async callback fetches the secret, retrying once with a trailing slash — the classic NC url-hash mismatch), get-shared-secret always 403 (Cairn never initiates), and the CardDAV system address book at remote.php/dav/addressbooks/system/system/system — REPORT sync-collection with per-server sync tokens, 404 tombstones via a cumulative Gone list (an opted-out user is actively deleted from the NC side, not just omitted), per-user vCards carrying CLOUD:user@instance.
  • Relationship-derived per-(user, server) listing exactly as decided: override wins both ways, then contact → sent → accepted; merely received never lists; default not listed; candidates enumerated from our own state only (no IdP enumeration).
  • UI: admin Directory tab (visibility only), Status row "Nextcloud & ownCloud compat", legacy product sub-line, sharpened legacy Remove confirm, and the "Where you appear" card in Federated → Contacts with the derivation always named and one link per row. i18n ×4. One mockup deviation, recorded: the admin tally shows the entries count without "of N users" — under OIDC the instance cannot enumerate its user total honestly.
  • Docs: handbook section incl. the no-trailing-slash note and the server-wall granularity. Audit verb ocm-directory-override.

Dogfood test path (Nikola): on Nextcloud → Settings → Federation, add files-bao.c0rdyceps.ch (no trailing slash). The add should now be accepted; the row goes green after NC's background job completes the handshake (cron — may take a few minutes) and the first address-book sync runs (occ federation:sync-addressbooks forces it). After that, nikola-test should autocomplete in NC's share dialog for Cordyborski — nikola-test is listed to that server by derivation (accepted shares). The Cairn side is visible under Administration → Open Cloud Mesh → Directory and, as a user, Federated → Contacts → Where you appear. Any handshake trouble now logs as trusted-servers … lines in the pod log.

Shipped in **v0.6.158**, live on both dogfoods — `cairn_build_info{version="v0.6.158"}` verified, and `https://files-bao.c0rdyceps.ch/status.php` already answers the probe JSON (active because nextcloud.c0rdyceps.ch sits on the allow-list as a legacy-dialect server). Built exactly to the locked decisions + approved rev-2 mockups, house TDD (backend red witnessed on the runner; wire formats pinned against the actual Nextcloud federation-app source): - **Compat surface** (`internal/ocm/directory.go`, mounted on the public mux, active only while a legacy server is allow-listed): `status.php`, OCS `request-shared-secret` (allow-list gated; async callback fetches the secret, retrying once with a trailing slash — the classic NC url-hash mismatch), `get-shared-secret` always 403 (Cairn never initiates), and the CardDAV system address book at `remote.php/dav/addressbooks/system/system/system` — REPORT sync-collection with per-server sync tokens, 404 tombstones via a cumulative Gone list (an opted-out user is actively deleted from the NC side, not just omitted), per-user vCards carrying `CLOUD:user@instance`. - **Relationship-derived per-(user, server) listing** exactly as decided: override wins both ways, then contact → sent → accepted; merely received never lists; default not listed; candidates enumerated from our own state only (no IdP enumeration). - **UI:** admin Directory tab (visibility only), Status row "Nextcloud & ownCloud compat", legacy product sub-line, sharpened legacy Remove confirm, and the "Where you appear" card in Federated → Contacts with the derivation always named and one link per row. i18n ×4. One mockup deviation, recorded: the admin tally shows the entries count without "of N users" — under OIDC the instance cannot enumerate its user total honestly. - **Docs:** handbook section incl. the no-trailing-slash note and the server-wall granularity. Audit verb `ocm-directory-override`. **Dogfood test path (Nikola):** on Nextcloud → Settings → Federation, add `files-bao.c0rdyceps.ch` (no trailing slash). The add should now be accepted; the row goes green after NC's background job completes the handshake (cron — may take a few minutes) and the first address-book sync runs (`occ federation:sync-addressbooks` forces it). After that, nikola-test should autocomplete in NC's share dialog for Cordyborski — nikola-test is listed to that server by derivation (accepted shares). The Cairn side is visible under Administration → Open Cloud Mesh → Directory and, as a user, Federated → Contacts → Where you appear. Any handshake trouble now logs as `trusted-servers …` lines in the pod log.
Author
Owner

Live end-to-end confirmation (2026-09-11), after v0.6.159–161 follow-ups:

19:59:42 GET  /ocs-provider/ → 401 (NC discovery probe, falls back to defaults)
19:59:42 REPORT /remote.php/dav/addressbooks/system/system/system → 207
19:59:42 GET  /remote.php/dav/addressbooks/system/system/system/nikola-test.vcf → 200

Nextcloud completed the handshake (Cairn Directory tab: verified), ran its address-book sync, and downloaded exactly one vCard — the single user with a derived relationship to that server. Row green on the NC side; nikola-test autocompletes in its share dialog. The sovereign per-(user, server) directory works as designed against a real Nextcloud 34.

Follow-up fixes that made it work, all shipped: v0.6.159 (Directory tab in ADM_TABS, neutral placeholder), v0.6.160 (in-app remove-server dialog), v0.6.161 (handshake callback follows the peer's OCS discovery — NC 34 serves the legacy GET /ocs/v2.php/cloud/shared-secret flavour, not the modern federation path). Operator-side prerequisites found during dogfooding, now in the handbook's troubleshooting picture: the NC instance must have overwrite.cli.url set to its public name (it identifies itself with it in the handshake), and NC only turns the row green after its first address-book sync, which is a slow timed job — occ federation:sync-addressbooks forces it.

**Live end-to-end confirmation (2026-09-11), after v0.6.159–161 follow-ups:** ``` 19:59:42 GET /ocs-provider/ → 401 (NC discovery probe, falls back to defaults) 19:59:42 REPORT /remote.php/dav/addressbooks/system/system/system → 207 19:59:42 GET /remote.php/dav/addressbooks/system/system/system/nikola-test.vcf → 200 ``` Nextcloud completed the handshake (Cairn Directory tab: verified), ran its address-book sync, and downloaded exactly **one** vCard — the single user with a derived relationship to that server. Row green on the NC side; nikola-test autocompletes in its share dialog. The sovereign per-(user, server) directory works as designed against a real Nextcloud 34. Follow-up fixes that made it work, all shipped: v0.6.159 (Directory tab in `ADM_TABS`, neutral placeholder), v0.6.160 (in-app remove-server dialog), v0.6.161 (handshake callback follows the peer's OCS discovery — NC 34 serves the legacy `GET /ocs/v2.php/cloud/shared-secret` flavour, not the modern federation path). Operator-side prerequisites found during dogfooding, now in the handbook's troubleshooting picture: the NC instance must have `overwrite.cli.url` set to its public name (it identifies itself with it in the handshake), and NC only turns the row green after its first address-book sync, which is a slow timed job — `occ federation:sync-addressbooks` forces it.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#455
No description provided.