Peering: inbound delivery semantics — placement, re-encryption, quota/audit interplay #104

Closed
opened 2026-08-04 11:04:13 +00:00 by Cordy · 2 comments
Owner

What happens after #102 accepts a stream: where the file lands and how it enters the recipient's world. This is where peering must compose with everything we already built — the design goal is that a received file is indistinguishable from one the recipient uploaded themselves.

Proposal: write through the FULL decorated storage stack with a synthesized recipient context (auth.User{Username: recipient} — custody providers only need the username), into /home/<recipient>/Inbox/<peerName>/<filename> (auto-created). That single decision buys, for free:

  • Re-encryption to the recipient's own key (keycloak-profile / openbao / deployment custody all work; lazy provisioning mints a key if the recipient never had one — verify this path explicitly for a user who has never logged in),
  • recipient-charged quota (pre-flight declared-size check in #102, enforced again on the real stream),
  • audit (peer-receive with source peer + sender-declared origin),
  • normal trash/retention/holds behavior afterwards.

Name collisions: never overwrite — suffix name (2).ext (like desktop conventions).

Open questions:

  • Flat-mode instances (perUserHomes off — every user sees the whole tree): /home/<user>/Inbox doesn't exist as a concept. Options: require perUserHomes for peering, or deliver to a shared /Inbox/<recipient>/ — recommend requiring perUserHomes in v1 and documenting it. ➤ confirm.
  • Group recipients (diagram allows group): delivery to /spaces/<group>/Inbox/? Deferred per #100's scope question — this issue implements users-only unless that changes.
  • Notification: how does the recipient learn a file arrived? v1 options: nothing (they find it in Inbox), a badge/toast in the web UI on next load, or email (we have no mailer — new dependency). Recommend: UI badge fed by a newest inbox mtime probe, no mailer. ➤ Nikola.
  • Seat accounting: a recipient who exists in the IdP but has NEVER signed in gets a file delivered — does that make them a "seen user" for licensing (#24 semantics count distinct authenticated usernames — delivery is not authentication, so recommendation: no seat consumed). Confirm.
  • Sender identity display: the wire (diagram) carries no sender USERNAME, only source instance. Do we add sender to the message so the recipient sees "from zeus321@Omega", or keep it instance-level only in v1? Metadata minimalism vs UX. ➤ Manuel.
  • Malware stance: no scanning in v1, document as the receiving admin's problem (same as uploads today) — OK?
What happens after #102 accepts a stream: where the file lands and how it enters the recipient's world. This is where peering must compose with everything we already built — the design goal is that a received file is indistinguishable from one the recipient uploaded themselves. **Proposal:** write through the FULL decorated storage stack with a synthesized recipient context (`auth.User{Username: recipient}` — custody providers only need the username), into `/home/<recipient>/Inbox/<peerName>/<filename>` (auto-created). That single decision buys, for free: - **Re-encryption to the recipient's own key** (keycloak-profile / openbao / deployment custody all work; lazy provisioning mints a key if the recipient never had one — verify this path explicitly for a user who has never logged in), - recipient-charged **quota** (pre-flight declared-size check in #102, enforced again on the real stream), - **audit** (`peer-receive` with source peer + sender-declared origin), - normal **trash/retention/holds** behavior afterwards. Name collisions: never overwrite — suffix `name (2).ext` (like desktop conventions). **Open questions:** - **Flat-mode instances** (`perUserHomes` off — every user sees the whole tree): `/home/<user>/Inbox` doesn't exist as a concept. Options: require perUserHomes for peering, or deliver to a shared `/Inbox/<recipient>/` — recommend requiring perUserHomes in v1 and documenting it. ➤ confirm. - Group recipients (diagram allows `group`): delivery to `/spaces/<group>/Inbox/`? Deferred per #100's scope question — this issue implements users-only unless that changes. - **Notification:** how does the recipient learn a file arrived? v1 options: nothing (they find it in Inbox), a badge/toast in the web UI on next load, or email (we have no mailer — new dependency). Recommend: UI badge fed by a `newest inbox mtime` probe, no mailer. ➤ Nikola. - **Seat accounting:** a recipient who exists in the IdP but has NEVER signed in gets a file delivered — does that make them a "seen user" for licensing (#24 semantics count distinct authenticated usernames — delivery is not authentication, so recommendation: no seat consumed). Confirm. - Sender identity display: the wire (diagram) carries no sender USERNAME, only source instance. Do we add `sender` to the message so the recipient sees "from zeus321@Omega", or keep it instance-level only in v1? Metadata minimalism vs UX. ➤ Manuel. - Malware stance: no scanning in v1, document as the receiving admin's problem (same as uploads today) — OK?
Author
Owner

Groundwork for this issue, recorded so it is not rediscovered. #123 is wired and green on main; CompleteTransfer currently refuses with STATUS_REFUSED because no Deliverer is registered, which is the gap this issue closes.

The contract that already exists

type Deliverer interface {
    Deliver(ctx context.Context, tr Transfer) error
}

Transfer carries Peer, TransferID, Recipient, FileName, Size, ContentHash, Sender, AgeIdentity, AgeRecipient, Capability, CreatedAt.

Note the signature has no reader. That is deliberate and it works: the implementation holds its own peering.Blobs (the same DirBlobs{Root: cfg.Peering.StagingDir} the BlobHandler writes to) and calls Open(tr.Peer, tr.TransferID). Wiring passes the same value to both. No signature change needed.

What the scout found

  • storage.FileInfo = {Name, Path, Size, ModTime, IsDir, ContentType, ETag}.
  • storage.Driver carries Delete, Mkdir (parents must exist — so /home/<r>/Inbox and /home/<r>/Inbox/<peer> must be created in order, not in one call), Rename, Copy. The Write signature was cut off by a head pipe and still needs confirming before coding.
  • internal/storage/encrypt/pq.go:40-42 already has the identity parser that handles both flavours — hybrid first, X25519 fallback. That is exactly what tr.AgeIdentity needs, and peering mints hybrids, so reuse it rather than calling age.ParseHybridIdentity directly.
  • internal/storage/encrypt/crypter.go:55 already does age.Decrypt(r, identities...). Same pattern applies here.
  • storage.ErrReadOnly / ErrHeld / ErrRetention exist and will surface naturally through the decorated stack — they should be mapped to a CompleteResponse detail rather than swallowed.

Plan

  1. Decrypt the staged blob with tr.AgeIdentity, streaming — never buffer, the #5 OOM lesson applies to WAN uploads verbatim.
  2. Verify tr.ContentHash against the plaintext while it decrypts (a TeeReader into a sha256). Doing it as a separate pass would decrypt twice. Mismatch → STATUS_HASH_MISMATCH, discard the blob, deliver nothing.
  3. Write through the full decorated stack (not unscopedStore's base) into /home/<recipient>/Inbox/<peer>/<file_name>, so quota, audit, trash, retention and holds apply with no new code, and re-encryption to the recipient's own key happens automatically because the encrypt decorator is in the chain.
  4. Collision policy on an existing name is undecided — suffix, overwrite, or refuse. Suffixing is the least destructive and matches what a mail client does.
  5. Blobs.Discard on both success and terminal failure; Registry.DropTransfer on success only (already called by CompleteTransfer).

Open

  • Confirm Driver.Write's exact signature before coding.
  • The Inbox path is invented by us, not by the user, so file_name is already validated as a single segment by PrepareTransfer — but the peer name also becomes a path component here and is only validated at registration. Worth re-checking validPeerName covers path-hostile input.

Scaffold .forgejo/workflows/driver-scout.yml is still in the repo — its head pipe returned 141 so the self-delete never ran. Remove it with the next workflow.

Groundwork for this issue, recorded so it is not rediscovered. #123 is wired and green on `main`; `CompleteTransfer` currently refuses with `STATUS_REFUSED` because no `Deliverer` is registered, which is the gap this issue closes. ## The contract that already exists ```go type Deliverer interface { Deliver(ctx context.Context, tr Transfer) error } ``` `Transfer` carries `Peer`, `TransferID`, `Recipient`, `FileName`, `Size`, `ContentHash`, `Sender`, `AgeIdentity`, `AgeRecipient`, `Capability`, `CreatedAt`. Note the signature has **no reader**. That is deliberate and it works: the implementation holds its own `peering.Blobs` (the same `DirBlobs{Root: cfg.Peering.StagingDir}` the `BlobHandler` writes to) and calls `Open(tr.Peer, tr.TransferID)`. Wiring passes the same value to both. No signature change needed. ## What the scout found - **`storage.FileInfo`** = `{Name, Path, Size, ModTime, IsDir, ContentType, ETag}`. - **`storage.Driver`** carries `Delete`, `Mkdir` (*parents must exist* — so `/home/<r>/Inbox` and `/home/<r>/Inbox/<peer>` must be created in order, not in one call), `Rename`, `Copy`. The `Write` signature was cut off by a `head` pipe and still needs confirming before coding. - **`internal/storage/encrypt/pq.go:40-42`** already has the identity parser that handles both flavours — hybrid first, X25519 fallback. That is exactly what `tr.AgeIdentity` needs, and peering mints hybrids, so reuse it rather than calling `age.ParseHybridIdentity` directly. - **`internal/storage/encrypt/crypter.go:55`** already does `age.Decrypt(r, identities...)`. Same pattern applies here. - **`storage.ErrReadOnly` / `ErrHeld` / `ErrRetention`** exist and will surface naturally through the decorated stack — they should be mapped to a `CompleteResponse` detail rather than swallowed. ## Plan 1. Decrypt the staged blob with `tr.AgeIdentity`, streaming — never buffer, the #5 OOM lesson applies to WAN uploads verbatim. 2. **Verify `tr.ContentHash` against the plaintext while it decrypts** (a `TeeReader` into a sha256). Doing it as a separate pass would decrypt twice. Mismatch → `STATUS_HASH_MISMATCH`, discard the blob, deliver nothing. 3. Write through the **full decorated stack** (not `unscopedStore`'s base) into `/home/<recipient>/Inbox/<peer>/<file_name>`, so quota, audit, trash, retention and holds apply with no new code, and re-encryption to the recipient's own key happens automatically because the encrypt decorator is in the chain. 4. Collision policy on an existing name is undecided — suffix, overwrite, or refuse. Suffixing is the least destructive and matches what a mail client does. 5. `Blobs.Discard` on both success and terminal failure; `Registry.DropTransfer` on success only (already called by `CompleteTransfer`). ## Open - Confirm `Driver.Write`'s exact signature before coding. - The `Inbox` path is invented by us, not by the user, so `file_name` is already validated as a single segment by `PrepareTransfer` — but the **peer name** also becomes a path component here and is only validated at registration. Worth re-checking `validPeerName` covers path-hostile input. Scaffold `.forgejo/workflows/driver-scout.yml` is still in the repo — its `head` pipe returned 141 so the self-delete never ran. Remove it with the next workflow.
Author
Owner

Implemented — closing

internal/peering/deliver.go, merged and shipped in v0.4.2. FileDeliverer is registered in cmd/cairnd, so CompleteTransfer now delivers instead of refusing.

Placement and the decorated stack

Delivery writes through unscopedStore — the full decorated stack minus the per-user scope decorator, which is exactly right because delivery addresses absolute /home/<recipient>/Inbox/<peer>/ paths rather than the caller's virtual root.

That single choice is what makes the rest of this issue's scope free: quota, audit, trash, retention, holds and re-encryption to the recipient's own key all apply with no new code, because they are decorators in that chain. Nothing here special-cases peering.

Mkdir requires parents to exist, so /home/<r>, /home/<r>/Inbox and /home/<r>/Inbox/<peer> are created top-down; an already-existing directory is not an error worth failing a delivery over.

Hash verification, and why it happens here

The plaintext hash is verified while the stream decrypts, via a TeeReader into a sha256. Doing it as a separate pass would decrypt twice. This is also why verification lives in the deliverer rather than in CompleteTransfer — decrypting is delivery.

The ordering problem this creates is handled explicitly: the hash cannot be known until the stream ends, and the stream is not buffered (the #5 OOM lesson applies verbatim to WAN uploads). So delivery writes to a dotted .part file and only renames it into place once the plaintext verifies. A corrupt transfer never appears under its real name, and the part file is deleted on mismatch — there is a test asserting nothing at all is left behind.

Path safety

Peer, recipient and file name all become path components here. file_name was already validated at PrepareTransfer, but the peer name was notvalidPeerName permits ., so .. was a registrable peer name that would have escaped the inbox. All three are re-validated at the point of use, with a test proving .. is refused and storage is never touched.

Collision policy

A second meme.jpg becomes meme-1.jpg. This was the open question in my earlier brief; I took suffix rather than overwrite or refuse, because overwriting someone's file because a peer reused a name is not a defensible default, and refusing turns a normal situation into a support ticket. Falls back to the transfer id after 999 collisions rather than looping.

8 tests including the end-to-end loopback, which moves a file sender → receiver in one process against the real Service, BlobHandler and FileDeliverer.

## Implemented — closing `internal/peering/deliver.go`, merged and shipped in v0.4.2. `FileDeliverer` is registered in `cmd/cairnd`, so `CompleteTransfer` now delivers instead of refusing. ### Placement and the decorated stack Delivery writes through **`unscopedStore`** — the full decorated stack *minus* the per-user scope decorator, which is exactly right because delivery addresses absolute `/home/<recipient>/Inbox/<peer>/` paths rather than the caller's virtual root. That single choice is what makes the rest of this issue's scope free: quota, audit, trash, retention, holds and **re-encryption to the recipient's own key** all apply with no new code, because they are decorators in that chain. Nothing here special-cases peering. `Mkdir` requires parents to exist, so `/home/<r>`, `/home/<r>/Inbox` and `/home/<r>/Inbox/<peer>` are created top-down; an already-existing directory is not an error worth failing a delivery over. ### Hash verification, and why it happens *here* The plaintext hash is verified **while the stream decrypts**, via a `TeeReader` into a sha256. Doing it as a separate pass would decrypt twice. This is also why verification lives in the deliverer rather than in `CompleteTransfer` — decrypting *is* delivery. The ordering problem this creates is handled explicitly: the hash cannot be known until the stream ends, and the stream is not buffered (the #5 OOM lesson applies verbatim to WAN uploads). So delivery writes to a dotted `.part` file and only renames it into place once the plaintext verifies. **A corrupt transfer never appears under its real name**, and the part file is deleted on mismatch — there is a test asserting nothing at all is left behind. ### Path safety Peer, recipient and file name all become path components here. `file_name` was already validated at `PrepareTransfer`, but the **peer name was not** — `validPeerName` permits `.`, so `..` was a registrable peer name that would have escaped the inbox. All three are re-validated at the point of use, with a test proving `..` is refused and storage is never touched. ### Collision policy A second `meme.jpg` becomes `meme-1.jpg`. This was the open question in my earlier brief; I took **suffix** rather than overwrite or refuse, because overwriting someone's file because a peer reused a name is not a defensible default, and refusing turns a normal situation into a support ticket. Falls back to the transfer id after 999 collisions rather than looping. 8 tests including the end-to-end loopback, which moves a file sender → receiver in one process against the real `Service`, `BlobHandler` and `FileDeliverer`.
Cordy closed this issue 2026-08-06 02:30:14 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#104
No description provided.