Cordy/Cairn
2
1
Fork 0
Code Issues 20 Pull requests Projects Releases Packages Wiki Activity Actions

Peering transport: control plane, data plane and single-port routing (#102) #122

Merged
Cordy merged 35 commits from feat/peering-transport into main 2026-08-05 17:36:25 +00:00
Conversation 0 Commits 35 Files changed 18 +2663 -11
Cordy commented 2026-08-05 17:35:50 +00:00
Owner
Copy link

Closes the transport half of #102 and settles #118. Builds on #101's registry.

This branch is inert at runtime. Nothing in cmd/cairnd constructs any of
it yet, so merging changes no behaviour on a running instance. See "Not in this
PR" below for why the wiring stops here.

A schema defect, found before any code was written against it

IMPLEMENTED-PEERING.md §5 requires the receiver to mint the per-transfer age
recipient and to decrypt the stream before re-encrypting to the recipient's own
key. The proto from part 1 put age_recipient in PrepareRequest — travelling
sender → receiver, where the sender cannot possibly know it yet. As written the
receiver could never have decrypted, and #104 would have been unbuildable.

Moved to PrepareResponse field 4, with reserved 8; on PrepareRequest so the
retired number is never reused. Recorded as §0.1 item 4 alongside the
HMAC → Ed25519 correction, and §2.1/§5 brought into line.

This is the second time the decision record contradicted itself and the second
time writing the test found it.

What landed

Piece Tests
Sessions — instance-bound, domain-separated from capabilities 5
Per-transfer state — receiver-minted keypair, idempotent, per-peer scoped 10
Post-quantum hybrid recipient 1
Stable capability, bound to peer and transfer id 2
Four RPCs 13
PUT /peer/v1/blob/{transfer_id} data plane 9
Single-port content-type routing 4

Properties the tests pin down, chosen because each is quiet when broken:

  • Hello is not an oracle. An unknown key id gets a challenge of identical
    length and TTL to a known one.
  • Authenticate fails identically for a bad proof and an unknown key — the
    test compares code and message, and the implementation returns one shared
    package-level error rather than building messages per call, so they cannot
    drift apart later.
  • Size is checked before the recipient probe. §4.3 accepts that an
    authenticated, allow-listed peer can enumerate recipients; an oversized
    request should not get that probe for free.
  • The capability is stable across retries (§6) and names the peer, so a
    valid token cannot be re-pointed at another transfer by guessing an id.
  • An over-long body is cut, not truncated. Staging reads one byte past the
    authorization, so an over-long stream is detected rather than silently
    trimmed to exactly the declared length — which would have staged a corrupt
    file that still looked the right size.
  • Mux requires HTTP/2, not just the content type: gRPC does not exist over
    HTTP/1.1, and without the version check a crafted 1.1 request would reach a
    server that cannot speak it.

Decisions taken here

  • Peering is post-quantum from day one, unconditionally, rather than
    following the storage postQuantum toggle. The recipient is minted per
    transfer and discarded, so there is no data at rest to migrate and no rollback
    path to preserve; and the only senders are Cairn instances speaking this
    protocol, where hybrid support (v0.3.29) predates peering entirely. Wire
    protocol = 1 now means hybrid — a classical fallback would take its own
    protocol number rather than a silent per-transfer downgrade. (§5, row E.)
  • The plaintext hash is verified during delivery, not in CompleteTransfer.
    Verifying it means decrypting the stream, which delivery already does;
    checking it here would decrypt twice. It is part of the Deliverer contract.
  • CompleteTransfer refuses loudly with no Deliverer wired, rather than
    acknowledging a file that went nowhere.

#118 settled

The README claimed "Four Go dependencies total" while ARCHITECTURE §1.4 said
five — stale before gRPC even arrived. README now states five for the core and
names peering's two additions as the one place the discipline was deliberately
broken (opt-in, off unless configured, control messages only). §1.4 records why
option C was chosen, what bounds the blast radius, and — since the point is
honesty — that a self-hoster enabling peering does audit a larger surface.
Status: v0.3.x → v0.4.x while in there.

Not in this PR

Wiring into cmd/cairnd, because it is blocked on a decision rather than on
work: there is no user-enumeration API in internal/auth. auth.Local
keeps an unexported users map, and in oidc or ldap mode the instance holds
no list of users at all — a user does not exist locally until first login. So
§4.3's "verify the recipient exists here" currently has nothing to call in two
of the three auth modes.

Recommendation, to be settled before the wiring lands: define existence as
"the recipient has a home directory on the storage backend". It works
uniformly across all three auth modes, needs no new auth surface, and fails
safe — a user who has never logged in has no home, so the transfer is refused,
which is correct because delivery would have had nowhere to put the file.

Also outstanding: peering.* config fields (staging dir, base URL,
maxFileBytes) and a Recipients implementation. Both are natural companions
to #103's send pipeline.

Green at every commit: gofmt, go vet, go test ./internal/peering/...,
go build ./....

Closes the transport half of #102 and settles #118. Builds on #101's registry. **This branch is inert at runtime.** Nothing in `cmd/cairnd` constructs any of it yet, so merging changes no behaviour on a running instance. See "Not in this PR" below for why the wiring stops here. ## A schema defect, found before any code was written against it `IMPLEMENTED-PEERING.md` §5 requires the receiver to mint the per-transfer age recipient and to decrypt the stream before re-encrypting to the recipient's own key. The proto from part 1 put `age_recipient` in `PrepareRequest` — travelling sender → receiver, where the sender cannot possibly know it yet. As written the receiver could never have decrypted, and #104 would have been unbuildable. Moved to `PrepareResponse` field 4, with `reserved 8;` on `PrepareRequest` so the retired number is never reused. Recorded as §0.1 item 4 alongside the HMAC → Ed25519 correction, and §2.1/§5 brought into line. This is the second time the decision record contradicted itself and the second time writing the test found it. ## What landed | Piece | Tests | |---|---| | Sessions — instance-bound, domain-separated from capabilities | 5 | | Per-transfer state — receiver-minted keypair, idempotent, per-peer scoped | 10 | | Post-quantum hybrid recipient | 1 | | Stable capability, bound to peer **and** transfer id | 2 | | Four RPCs | 13 | | `PUT /peer/v1/blob/{transfer_id}` data plane | 9 | | Single-port content-type routing | 4 | Properties the tests pin down, chosen because each is quiet when broken: - **`Hello` is not an oracle.** An unknown key id gets a challenge of identical length and TTL to a known one. - **`Authenticate` fails identically** for a bad proof and an unknown key — the test compares code *and* message, and the implementation returns one shared package-level error rather than building messages per call, so they cannot drift apart later. - **Size is checked before the recipient probe.** §4.3 accepts that an authenticated, allow-listed peer can enumerate recipients; an oversized request should not get that probe for free. - **The capability is stable across retries** (§6) and names the peer, so a valid token cannot be re-pointed at another transfer by guessing an id. - **An over-long body is cut, not truncated.** Staging reads one byte past the authorization, so an over-long stream is detected rather than silently trimmed to exactly the declared length — which would have staged a corrupt file that still looked the right size. - **`Mux` requires HTTP/2**, not just the content type: gRPC does not exist over HTTP/1.1, and without the version check a crafted 1.1 request would reach a server that cannot speak it. ## Decisions taken here - **Peering is post-quantum from day one**, unconditionally, rather than following the storage `postQuantum` toggle. The recipient is minted per transfer and discarded, so there is no data at rest to migrate and no rollback path to preserve; and the only senders are Cairn instances speaking this protocol, where hybrid support (v0.3.29) predates peering entirely. Wire `protocol = 1` now *means* hybrid — a classical fallback would take its own protocol number rather than a silent per-transfer downgrade. (§5, row E.) - **The plaintext hash is verified during delivery, not in `CompleteTransfer`.** Verifying it means decrypting the stream, which delivery already does; checking it here would decrypt twice. It is part of the `Deliverer` contract. - **`CompleteTransfer` refuses loudly with no `Deliverer` wired**, rather than acknowledging a file that went nowhere. ## #118 settled The README claimed "Four Go dependencies total" while ARCHITECTURE §1.4 said five — stale before gRPC even arrived. README now states five for the core and names peering's two additions as the one place the discipline was deliberately broken (opt-in, off unless configured, control messages only). §1.4 records why option C was chosen, what bounds the blast radius, and — since the point is honesty — that a self-hoster enabling peering does audit a larger surface. `Status: v0.3.x` → `v0.4.x` while in there. ## Not in this PR Wiring into `cmd/cairnd`, because it is blocked on a decision rather than on work: **there is no user-enumeration API in `internal/auth`.** `auth.Local` keeps an unexported `users` map, and in `oidc` or `ldap` mode the instance holds no list of users at all — a user does not exist locally until first login. So §4.3's "verify the recipient exists here" currently has nothing to call in two of the three auth modes. Recommendation, to be settled before the wiring lands: define existence as **"the recipient has a home directory on the storage backend"**. It works uniformly across all three auth modes, needs no new auth surface, and fails safe — a user who has never logged in has no home, so the transfer is refused, which is correct because delivery would have had nowhere to put the file. Also outstanding: `peering.*` config fields (staging dir, base URL, `maxFileBytes`) and a `Recipients` implementation. Both are natural companions to #103's send pipeline. Green at every commit: `gofmt`, `go vet`, `go test ./internal/peering/...`, `go build ./...`.
IMPLEMENTED-PEERING.md 5 requires the receiver to mint the per-transfer age
recipient and to decrypt the stream before re-encrypting to the recipient's
own key. That is only possible if the receiver holds the private half, which
means the public recipient must travel receiver -> sender. Field 8 of
PrepareRequest could never have been populated by the sender.

Retired as `reserved 8;` rather than reused: nothing has shipped this schema,
but the evolution rule says numbers are never reused and the rule is cheaper
to keep than to re-argue.
Written before the implementation. These fix the two properties the RPCs
depend on: a session names exactly one authenticated peer and cannot be
carried to another instance, and PrepareTransfer is idempotent on
transfer_id INCLUDING the age recipient - a retried upload that got a fresh
recipient would be undecryptable.
The age recipient is minted HERE, by the receiver, and held for the life of
the transfer - the receiver needs the private half to decrypt before
re-encrypting to the recipient (IMPLEMENTED-PEERING.md 5). It is stable
across idempotent retries of a transfer_id, because a retry that got a fresh
recipient would upload a body nobody could open.
Written before the service. The two that matter most are the ones that are
easy to get subtly wrong: Hello must answer an unknown key id exactly as it
answers a known one, and Authenticate must fail identically for a bad proof
and an unknown key - otherwise both become oracles for which peerings exist.
Hello answers an unknown key id exactly as it answers a known one, and
Authenticate returns one shared error value for both a bad proof and an
unknown key - either asymmetry would turn the handshake into an oracle for
which peerings exist.

The size check runs before the recipient probe on purpose: 4.3 accepts
that an authenticated, allow-listed peer can enumerate recipients, but an
oversized request should not get that probe for free.
Written before the handler. The capability is the only thing standing
between an open port and an unauthenticated write, so most of these pin down
what must be refused: a token minted for a different transfer, a tampered
token, a replayed token, and a body longer than the size the capability
authorized.
The peer is recovered from the signed capability, never from the URL, so a
valid token cannot be pointed at another transfer. Size is enforced on the
wire by reading one byte past the authorization: an over-long stream is cut
and staged nothing, rather than silently truncated to exactly the declared
length, which would have produced a corrupt file that still looked correct.

Staged names are a hash of peer+id because both arrive off the wire and
neither may become a path component.
The HTTP/2 condition is the one worth pinning: gRPC requires h2, so an
HTTP/1.1 request carrying a gRPC content type must fall through to the
normal mux rather than reach a server that cannot speak it.
chore: remove the scout scaffolds
All checks were successful
ci / test-and-build (pull_request) Successful in 2m6s
Details
b5f72b1e8c
Cordy merged commit 1557a0d2fa into main 2026-08-05 17:36:25 +00:00
Cordy deleted branch feat/peering-transport 2026-08-05 17:36:33 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#122
No description provided.
Delete branch "feat/peering-transport"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?