Cordy/Cairn

Fork 0

Peering delivery can't encrypt to a non-session recipient on per-user custody (blocks #106) — deep-dive #237

New issue

Closed

opened 2026-08-14 18:02:13 +00:00 by Cordy · 2 comments

Cordy commented

2026-08-14 18:02:13 +00:00

Owner

Deep-research / deep-dive. Rank permanent, sound designs; weight correctness, security, and cross-custody consistency. Do not down-weight an option for being harder to implement.

What surfaced

2026-08-14 A↔B dogfood (files/enc ↔ files-bao/openbao), first real end-to-end transfer. Control plane is fully working now — handshake (#102) over the Caddy h2c path, send (#103), and the recipient picker (#125) all succeed. The transfer then fails at delivery on the receiver:

peering: STATUS_REFUSED: peering: write: encrypt: no authenticated user in context

Root cause

FileDeliverer.Deliver (internal/peering/deliver.go) decrypts the staged blob with the per-transfer identity this instance minted, then writes the plaintext through the full decorated storage stack — d.Sink.Write(ctx, …) — precisely so it's re-encrypted to the recipient's own key (IMPLEMENTED-PEERING.md §5: "re-encryption to the recipient's own key … applies with no new code").

But the re-encryption decorator derives the target key from the authenticated user in the context, and the ctx here is the peering CompleteTransfer RPC context — which has no authenticated user. The peer authenticated over gRPC; the recipient user has no session on the receiving instance. So the encrypt decorator throws no authenticated user in context, and CompleteTransfer returns STATUS_REFUSED.

Impact

Peering file delivery does not work on per-user-encrypted instances — i.e. the primary custody modes (keycloak-profile, openbao). The §5 "no new code" promise only holds where the encryption target is session-independent (instance-static / recovery identity). For real per-user custody, delivery needs a capability it doesn't have: resolve and encrypt to the recipient's key without the recipient's session. Handshake + discovery are proven; this is the remaining gate on #106 being genuinely usable rather than handshake-complete.

Deep-dive questions

Resolve the recipient's write recipient without a session. Encryption needs only the recipient's public age recipient. Per mode:
- keycloak-profile: read the recipient's age recipient from their Keycloak profile via the admin client (the one #96/#178/#226 already use). Is the public recipient stored in the profile, or only the private identity (→ we'd be reading sensitive material just to derive the public half)?
- openbao: read the recipient's key from OpenBao KV via the app token, not the user's.
- LDAP / static / recovery-identity: what is the correct target?
  Define one custody-agnostic "resolve recipient write-recipient" seam.
Delivery context / scope. The deliverer must produce a write path the encrypt+scope decorators accept as writing for the recipient — a synthesized auth.User carrying the resolved recipient, or a delivery-scoped Sink built for that user. Writing as another user is a genuinely new capability; bound it tightly to the delivery path and reason about its blast radius.
Recovery / escrow. Should delivered files also be encrypted to the deployment recovery identity (so recovery + audited recovery reads work), matching normal user writes? Keep consistent with the existing per-user + recovery scheme.
Failure semantics. Today recipient-key-resolution failure is a blunt STATUS_REFUSED. Distinguish "recipient has no provisioned key" / "IdP admin client absent" / "unknown recipient" so sender and the receiver's audit can tell them apart. §4.3 already accepts recipient existence as known.
Align with the resident check (#123). Recipients.Exists gates PrepareTransfer on residency. Does "resident" guarantee a provisioned encryption key? If a resident user can still lack one, PrepareTransfer passes but delivery fails — the two checks must agree, so anything accepted at prepare can always be delivered.

Deliverable

A design for recipient-key resolution across custody modes + the delivery-context change, then implement. Note the sharp edge: this is the difference between "peering handshakes" and "peering actually moves files" on the modes people will run.

Refs: #104, #102, #103, #106, #123, #125; IMPLEMENTED-PEERING.md §5; internal/peering/deliver.go. Blocks #106.

**Deep-research / deep-dive.** Rank permanent, sound designs; weight correctness, security, and cross-custody consistency. Do **not** down-weight an option for being harder to implement. ## What surfaced 2026-08-14 A↔B dogfood (files/enc ↔ files-bao/openbao), **first real end-to-end transfer**. Control plane is fully working now — handshake (#102) over the Caddy h2c path, send (#103), and the recipient picker (#125) all succeed. The transfer then fails at **delivery on the receiver**: ``` peering: STATUS_REFUSED: peering: write: encrypt: no authenticated user in context ``` ## Root cause `FileDeliverer.Deliver` (`internal/peering/deliver.go`) decrypts the staged blob with the per-transfer identity this instance minted, then writes the plaintext through the **full decorated storage stack** — `d.Sink.Write(ctx, …)` — precisely so it's re-encrypted to the recipient's own key (IMPLEMENTED-PEERING.md §5: *"re-encryption to the recipient's own key … applies with no new code"*). But the re-encryption decorator derives the **target key from the authenticated user in the context**, and the `ctx` here is the peering `CompleteTransfer` RPC context — which has **no authenticated user**. The *peer* authenticated over gRPC; the *recipient user* has no session on the receiving instance. So the encrypt decorator throws `no authenticated user in context`, and `CompleteTransfer` returns `STATUS_REFUSED`. ## Impact **Peering file delivery does not work on per-user-encrypted instances** — i.e. the primary custody modes (keycloak-profile, openbao). The §5 "no new code" promise only holds where the encryption target is session-independent (instance-static / recovery identity). For real per-user custody, delivery needs a capability it doesn't have: resolve and encrypt to the **recipient's** key **without the recipient's session**. Handshake + discovery are proven; this is the remaining gate on #106 being genuinely usable rather than handshake-complete. ## Deep-dive questions 1. **Resolve the recipient's write recipient without a session.** Encryption needs only the recipient's *public* age recipient. Per mode: - **keycloak-profile:** read the recipient's age recipient from their Keycloak profile via the **admin client** (the one #96/#178/#226 already use). Is the public recipient stored in the profile, or only the private identity (→ we'd be reading sensitive material just to derive the public half)? - **openbao:** read the recipient's key from OpenBao KV via the **app token**, not the user's. - **LDAP / static / recovery-identity:** what is the correct target? Define one custody-agnostic "resolve recipient write-recipient" seam. 2. **Delivery context / scope.** The deliverer must produce a write path the encrypt+scope decorators accept as *writing for the recipient* — a synthesized `auth.User` carrying the resolved recipient, or a delivery-scoped Sink built for that user. Writing **as another user** is a genuinely new capability; bound it tightly to the delivery path and reason about its blast radius. 3. **Recovery / escrow.** Should delivered files also be encrypted to the deployment **recovery identity** (so recovery + audited recovery reads work), matching normal user writes? Keep consistent with the existing per-user + recovery scheme. 4. **Failure semantics.** Today recipient-key-resolution failure is a blunt `STATUS_REFUSED`. Distinguish "recipient has no provisioned key" / "IdP admin client absent" / "unknown recipient" so sender and the receiver's audit can tell them apart. §4.3 already accepts recipient existence as known. 5. **Align with the resident check (#123).** `Recipients.Exists` gates PrepareTransfer on residency. Does "resident" guarantee a provisioned encryption key? If a resident user can still lack one, PrepareTransfer passes but delivery fails — the two checks must agree, so anything accepted at prepare can always be delivered. ## Deliverable A design for recipient-key resolution across custody modes + the delivery-context change, then implement. Note the sharp edge: this is the difference between "peering handshakes" and "peering actually moves files" on the modes people will run. Refs: #104, #102, #103, #106, #123, #125; IMPLEMENTED-PEERING.md §5; `internal/peering/deliver.go`. **Blocks #106.**

Cordy commented

2026-08-15 12:06:41 +00:00

Author

Owner

Deep-dive from the code (2026-08-14) — the question is narrower than framed

Read the encryption stack end to end (encrypt.go, crypter.go, keycloak.go, openbao.go, cryptosvc.go, auth/context.go). The architecture already supports encrypting to a non-session recipient — delivery simply never puts the recipient into the write context. Ground truth, then ranked options.

What the code actually does

Encryption is a storage decorator (encrypt.Driver). Write → KeyProvider.Recipients(ctx) → encrypt body to those age recipients. Open → decrypt with Identities(ctx).
Every KeyProvider resolves keys by the username in context, via deployment/admin credentials — never the user's own session:
- KeycloakKeys.keysFor: u := auth.UserFrom(ctx); nil → errors.New("encrypt: no authenticated user in context") (our exact error). Otherwise a service-account admin client (client_credentials) looks the user up by name and reads/provisions their age identity from the profile attribute.
- OpenBaoKeys.keysFor: same shape; resolves users/<username> from the KV mount with the app Vault token.
  Both need only u.Username, both auto-append the deployment Recovery recipient, both lazily provision an identity if the principal has none.
"Operate as an arbitrary named user" is already a sanctioned first-class capability: the crypto role (cryptosvc) does exactly it — X-Cairn-User: <name> → auth.ContextWithUser(ctx, &auth.User{Username: name}) → resolve that user's keys ("it decrypts on request for any username the caller names, which is exactly the trust the API role already has"). The split-replica client forwards UserFrom(ctx).Username as X-Cairn-User, so anything that sets the context behaves identically in-process and against a --target=crypto replica.
The deliverer (deliver.go) was built to write through the full decorated stack ("Sink … must be the FULL decorated stack … re-encryption to the recipient's own key applies with no new code") — but calls d.Sink.Write(ctx, …) with the raw CompleteTransfer RPC context, which has no user. So it hits (2)'s nil branch.

The bug: delivery omits the recipient from the write context. Nothing else architecturally is missing.

Approaches (weighted security / stability / long-term correctness; quick-patch discounted)

A — Populate the recipient into the delivery context (recommended). In Deliver, before the Sink calls: ctx = auth.ContextWithUser(ctx, &auth.User{Username: tr.Recipient}). ContextWithUser is exported precisely "for callers that establish identity outside Middleware." Consequences, all correct:

encrypts to the recipient's key + deployment recovery (recipient-readable, admin-recoverable);
counts against the recipient's quota; subject to their holds; writes to /home/<recipient>/Inbox (personal path → user key, not group);
lazily provisions a key if the recipient never logged in — which also closes the #123 resident-vs-key gap: a resident recipient is always deliverable;
works unchanged in-process and via the crypto replica;
PQ-safe automatically (resolveKeyset mints/upgrades hybrid ML-KEM identities).
Not a patch — it's the design the deliverer already assumed (~2 lines + import). Security note: it makes the deliverer write as the recipient; that authority already exists at the API/crypto role and is bounded to tr.Recipient, itself gated by the #123 residency + allow-list check at PrepareTransfer.

B — New explicit KeyProvider.RecipientsFor(username) + delivery-only write path. More self-documenting (no synthetic user in context) but duplicates resolution, bypasses the crypto-replica seam, adds surface for no behavioral gain. Rejected.

C — Route delivery through cryptosvc with X-Cairn-User = recipient. Not a separate option — it's what A already does transparently when crypto is split. No action.

D — True client-side E2E (sender encrypts to the recipient's published public key). The zero-trust direction (age-to-published-recipient / MLS / Signal-style) where the receiving server never sees plaintext. This is a product-model change, not a delivery fix — Cairn is deliberately server-side transparent today (the code says so: "NOT client-side E2E … not protection against a compromised server"). Cross-instance E2E would need published recipient keys, a sender-side encryption pipeline, and forfeiting server-side features on delivered files. Worth a separate long-term product issue; out of scope for making #106 work under the stated model.

E — Pre-resolve recipient recipients at PrepareTransfer, thread to CompleteTransfer. No benefit over A, adds transfer state, receiver already has the resolver. Rejected.

Recommendation

A, plus two refinements: (1) a distinct failure status when recipient key resolution/provisioning fails (vs generic STATUS_REFUSED), so the sender can tell "unprovisionable recipient" from "delivery error"; (2) confirm the stack's audit doesn't double-log — the deliverer's peer-deliver stays canonical.

Industry framing: this is the standard KMS/server-side-encryption model (resolve any principal's key by identity, à la per-object DEKs) — the KeyProvider is the KMS seam and A stays squarely inside it. D is where zero-trust systems (MLS, Proton/Tresorit) sit; that's a deliberate future product decision, not this bug. Filing D separately.

## Deep-dive from the code (2026-08-14) — the question is narrower than framed Read the encryption stack end to end (`encrypt.go`, `crypter.go`, `keycloak.go`, `openbao.go`, `cryptosvc.go`, `auth/context.go`). The architecture **already** supports encrypting to a non-session recipient — delivery simply never puts the recipient into the write context. Ground truth, then ranked options. ### What the code actually does 1. Encryption is a storage decorator (`encrypt.Driver`). `Write` → `KeyProvider.Recipients(ctx)` → encrypt body to those age recipients. `Open` → decrypt with `Identities(ctx)`. 2. **Every KeyProvider resolves keys by the *username in context*, via deployment/admin credentials — never the user's own session:** - `KeycloakKeys.keysFor`: `u := auth.UserFrom(ctx)`; **nil → `errors.New("encrypt: no authenticated user in context")`** (our exact error). Otherwise a **service-account admin client** (`client_credentials`) looks the user up by name and reads/provisions their age identity from the profile attribute. - `OpenBaoKeys.keysFor`: same shape; resolves `users/<username>` from the KV mount with the **app Vault token**. Both need only `u.Username`, both auto-append the deployment **Recovery** recipient, both **lazily provision** an identity if the principal has none. 3. "Operate as an arbitrary named user" is already a sanctioned first-class capability: the crypto role (`cryptosvc`) does exactly it — `X-Cairn-User: <name>` → `auth.ContextWithUser(ctx, &auth.User{Username: name})` → resolve that user's keys (*"it decrypts on request for any username the caller names, which is exactly the trust the API role already has"*). The split-replica client forwards `UserFrom(ctx).Username` as `X-Cairn-User`, so anything that sets the context behaves identically in-process and against a `--target=crypto` replica. 4. The deliverer (`deliver.go`) was **built to write through the full decorated stack** ("Sink … must be the FULL decorated stack … re-encryption to the recipient's own key applies with no new code") — but calls `d.Sink.Write(ctx, …)` with the **raw CompleteTransfer RPC context**, which has no user. So it hits (2)'s nil branch. **The bug: delivery omits the recipient from the write context. Nothing else architecturally is missing.** ### Approaches (weighted security / stability / long-term correctness; quick-patch discounted) **A — Populate the recipient into the delivery context (recommended).** In `Deliver`, before the Sink calls: `ctx = auth.ContextWithUser(ctx, &auth.User{Username: tr.Recipient})`. `ContextWithUser` is exported *precisely* "for callers that establish identity outside Middleware." Consequences, all correct: - encrypts to the **recipient's** key + deployment recovery (recipient-readable, admin-recoverable); - counts against the **recipient's** quota; subject to their holds; writes to `/home/<recipient>/Inbox` (personal path → user key, not group); - **lazily provisions** a key if the recipient never logged in — which *also closes the #123 resident-vs-key gap*: a resident recipient is always deliverable; - works unchanged in-process **and** via the crypto replica; - PQ-safe automatically (`resolveKeyset` mints/upgrades hybrid ML-KEM identities). Not a patch — it's the design the deliverer already assumed (~2 lines + import). Security note: it makes the deliverer *write as* the recipient; that authority already exists at the API/crypto role and is bounded to `tr.Recipient`, itself gated by the #123 residency + allow-list check at PrepareTransfer. **B — New explicit `KeyProvider.RecipientsFor(username)` + delivery-only write path.** More self-documenting (no synthetic user in context) but duplicates resolution, bypasses the crypto-replica seam, adds surface for no behavioral gain. Rejected. **C — Route delivery through `cryptosvc` with `X-Cairn-User = recipient`.** Not a separate option — it's what A already does transparently when crypto is split. No action. **D — True client-side E2E (sender encrypts to the recipient's published public key).** The zero-trust direction (age-to-published-recipient / MLS / Signal-style) where the receiving server never sees plaintext. This is a **product-model change**, not a delivery fix — Cairn is deliberately *server-side transparent* today (the code says so: "NOT client-side E2E … not protection against a compromised server"). Cross-instance E2E would need published recipient keys, a sender-side encryption pipeline, and forfeiting server-side features on delivered files. Worth a **separate long-term product issue**; out of scope for making #106 work under the stated model. **E — Pre-resolve recipient recipients at PrepareTransfer, thread to CompleteTransfer.** No benefit over A, adds transfer state, receiver already has the resolver. Rejected. ### Recommendation **A**, plus two refinements: (1) a distinct failure status when recipient key resolution/provisioning fails (vs generic `STATUS_REFUSED`), so the sender can tell "unprovisionable recipient" from "delivery error"; (2) confirm the stack's audit doesn't double-log — the deliverer's `peer-deliver` stays canonical. Industry framing: this is the standard KMS/server-side-encryption model (resolve any principal's key by identity, à la per-object DEKs) — the KeyProvider is the KMS seam and A stays squarely inside it. **D** is where zero-trust systems (MLS, Proton/Tresorit) sit; that's a deliberate future product decision, not this bug. Filing D separately.

Cordy referenced this issue

2026-08-15 12:07:07 +00:00

Product direction: should cross-instance (peering) transfer be client-side E2E? (long-term) #238

Cordy referenced this issue

2026-08-15 13:09:55 +00:00

Product direction: should cross-instance (peering) transfer be client-side E2E? (long-term) #238

Cordy added this to the v0.7 — Closed beta (dogfood + hardening) milestone

2026-08-15 13:18:41 +00:00

forgejo-actions referenced this issue from a commit

2026-08-15 13:22:27 +00:00

#237 peering delivery: encrypt to the recipient (v0.6.23)

Cordy referenced this issue from a pull request that will close it,

2026-08-15 13:23:50 +00:00

#237 peering delivery: encrypt to the recipient (v0.6.23) #239

Cordy closed this issue

2026-08-15 13:25:37 +00:00

Cordy referenced this issue from a commit

2026-08-15 13:25:37 +00:00

#237 peering delivery: encrypt to the recipient (v0.6.23) (#239)

Cordy commented

2026-08-15 13:41:09 +00:00

Author

Owner

Verified live on v0.6.23 (2026-08-15)

Option A is deployed and the transfer that failed here now completes end to end.

Transfer: Andrej-karpathy-skills.md (5.4 KB), files-bao → nikola-test@files, status done at 13:32 — sitting directly above the two 2026-08-14 failures that read peering: STATUS_REFUSED: peering: write: encrypt: no authenticated user in context.
Authorization: via the cairn-admins group (no explicit user on the receivers list — group membership resolved live), confirming the fix works with group-based receivers.
Path: over the HTTPS/Caddy reverse proxy (the h2c matcher), not the LAN IP.
Delivery: the file landed in the recipient's /home/nikola-test/Inbox/bao/ on the files instance and renders in their listing — i.e. it was re-encrypted to nikola-test's own key (keycloak-profile custody, resolved by username via the admin client) and is decryptable by them.

Every layer of the fix is exercised: recipient context populated → encrypt decorator resolved the recipient's key by username → re-encrypted to recipient + recovery → delivered to their Inbox. Closing confirmed.

## Verified live on v0.6.23 (2026-08-15) Option A is deployed and the transfer that failed here now completes end to end. - **Transfer:** `Andrej-karpathy-skills.md` (5.4 KB), files-bao → `nikola-test@files`, status **done** at 13:32 — sitting directly above the two 2026-08-14 failures that read `peering: STATUS_REFUSED: peering: write: encrypt: no authenticated user in context`. - **Authorization:** via the `cairn-admins` **group** (no explicit user on the receivers list — group membership resolved live), confirming the fix works with group-based receivers. - **Path:** over the HTTPS/Caddy reverse proxy (the h2c matcher), not the LAN IP. - **Delivery:** the file landed in the recipient's `/home/nikola-test/Inbox/bao/` on the files instance and **renders in their listing** — i.e. it was re-encrypted to nikola-test's own key (keycloak-profile custody, resolved by username via the admin client) and is decryptable by them. Every layer of the fix is exercised: recipient context populated → encrypt decorator resolved the recipient's key by username → re-encrypted to recipient + recovery → delivered to their Inbox. Closing confirmed.

Cordy referenced this issue

2026-08-15 13:41:24 +00:00

Peering dogfood: second instance on k3s + live A↔B e2e #106

Cordy referenced this issue

2026-08-15 15:31:59 +00:00

Peering: send into a Space's shared inbox (Space as a peering recipient) #244