Peering: outbound send pipeline + transfer status #103
Labels
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Cordy/Cairn#103
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The sender half: a user picks a file, a registered peer (from #101's registry) and a recipient; cairnd streams the file out per the #102 protocol.
Pipeline: resolve the file through the caller's own scoped/decorated store (so decryption happens with THEIR keys and read-authorization is enforced by the existing stack — sending is a read) → open connection to the peer URL → handshake → stream chunked → confirm content hash → record outcome.
API sketch:
POST /api/v1/peering/send{path, peer, recipient}→ returns transfer id immediatelyGET /api/v1/peering/transfers→ caller's transfers (pending / streaming / done / failed+reason)Async by design: a multi-GB file over WAN cannot ride one browser request. Background job with status polling; job state in a JSON store (no-DB rule), survives restart as
failed(v1: no mid-flight resume, see #102).Audit event
peer-send(path, peer, recipient, outcome). Metrics: transfers started/succeeded/failed, bytes out.Open questions:
peering.userSendEnabledto restrict to admins — which default? ➤ Nikola (product).peering.maxConcurrentSends, per-stream rate cap) or trust the network? At minimum a concurrency cap, symmetric with #102's inbound cap.Open questions resolved
Most of this issue's open questions were written before the ADR landed, and
IMPLEMENTED-PEERING.mdalready answers them:Registry.CanSendagainst the caller's live groups (#96). The proposed globalpeering.userSendEnabledtoggle is superseded and should not be built.Legal holds (decided by Nikola, 2026-08-06)
A file under a legal hold may not be sent off-instance. If any hold covers the file or a parent folder, the send is refused.
This is a deliberate departure from current semantics rather than an inherited one, and worth stating clearly: the hold decorator blocks delete, overwrite and rename, but not reads — and sending is a read. So without an explicit check, a legal hold would be trivially defeated by peering the file to another instance. That is the opposite of what a hold is for, particularly for the public-sector persona (#59, #63).
Scoped as "at least for now" — a later refinement could distinguish a hold that freezes mutation from one that also forbids egress, but v1 treats every hold as forbidding both.
Implementation seam: the sender takes a
Held func(path string) boolrather than importing the hold store directly, sopeeringdoes not grow a dependency oninternal/storage/holdand the wiring adapts whatever the real accessor is called.Defaults taken (documented, not asked)
failedwith a reason; the user re-triggers. Keeps the state machine trivial, as the issue itself recommends.Also fixed while getting here
PrepareRequest.sizeis the plaintext size, but the wire carries age ciphertext, which is always larger. The data plane was capping the stream at exactlysizeand rejectingn != size— so every correctly-encrypted upload would have been truncated and then refused. It passed CI only because every test so far uploaded plaintext. Replaced withwireAllowance(size)(header + a tag per 64 KiB chunk) and the byte-count equality check dropped: integrity is the age AEAD and the plaintext content hash, both verified during delivery, not a length comparison over ciphertext.Implemented
Merged in PR #126 (pipeline) and #127 (API + wiring), shipped v0.4.2. Files:
internal/peering/{client,sender}.go,internal/api/peering_send.go.Client — the protocol half
Client.SenddrivesHello → Prove → Authenticate → PrepareTransfer → PUT → CompleteTransfer. Encryption streams through anio.Pipe, so a multi-gigabyte file is encrypted and sent as it is read and never buffered.Verified by a genuine in-process loopback: real
Serviceover bufconn, realBlobHandlerover httptest, realFileDelivererat the far end. That test is the first thing in the track that fails if any layer is wrong — handshake, capability, age-recipient direction, wire-size allowance, inbox path or hash check.Sender — authorization and the state machine
Every refusal is synchronous, before
Startreturns, so the caller gets a real error rather than a job that dies a second later:CanSend— the per-peering allow-listHeld(p) || Affected(p), so a file inside a held folder is blocked tooWritable == nilmeans space content cannot be sent at allTransfersis atomic JSON, no database. On open, anything stillpendingorstreamingbecomesfailed— v1 has no mid-flight resume, so a transfer that died with the process must not keep looking alive.ForUserscopes the list to its owner.API
Not admin-gated: sending is a per-peering decision, so an admin without an allow-list entry cannot send and a non-admin with one can.
/peering/peersreturns a projection, never the storedPeer— that struct carries the peering key, and returning it directly would hand a credential to every authenticated user. Tested.Two things found while building this
The data plane would have cut every real upload.
sizeis the plaintext size but the wire carries ciphertext. Detailed in the v0.4.2 changelog.A replayed transfer id would have delivered a second copy. §6's idempotency only covers in-flight retries; after successful delivery the record is dropped, so a sender whose
CompleteTransferresponse was lost — exactly what this issue's async job does on restart — would duplicate the file. AddedMarkDelivered/WasDeliveredwith a 24h TTL. At-least-once on the wire must not become at-least-twice on disk.Defaults taken, per the issue's own recommendations
No automatic retries, files only, concurrency cap symmetric with inbound. The "who may send" and "read-only space member" questions were superseded by §4.2 and are recorded above.