Peering: how does the receiver decide a recipient "exists"? (blocks the cmd/cairnd wiring) #123
Closed
opened 2026-08-05 17:37:08 +00:00 by Cordy
·
5 comments
No Branch/Tag specified
main
b12v07
b11v07
b10v07
ci-rc-publish
fx15-s3dry
b9v07
b8v07
bench742
b7v07
legal-texts-v0.2
b6v07
b5v07
b4v07
b3v07
sx1139
st742
b2v07
ix1102
fu1098
zp1097
wd1099
tr1100
qf1077
nc1070
ci1
rp1072
lk1073
ab1071
fi1074
si1070
ra1054
wo1056
oc1057
tf1055
sv1063
fg1059
st1058
fs929
mu1039
ls1033
cf975
lf835
dx1023
ml896
a11y2
av1022
r913
r7
r6
r5
r45
r23
r1
b10
b9
fix-b5
fix-b4
fix-b3
fix-b2
fix-b1
fix-922
fix-923
fix-911
fix-343b
batch-a
v0.6.413
v0.6.412
v0.6.411
v0.6.410
v0.6.409
v0.6.408
v0.6.407
v0.6.406
v0.6.405
v0.6.404
v0.6.403
v0.6.402
v0.6.401
v0.6.400
v0.6.399
v0.6.398
v0.6.397
v0.6.396
v0.6.395
v0.6.394
v0.6.393
v0.6.392
v0.6.391
v0.6.390
v0.6.389
v0.6.388
v0.6.387
v0.6.386
v0.6.385
v0.6.384
v0.6.383
v0.6.382
v0.6.381
v0.6.380
v0.6.379
v0.6.378
v0.6.377
v0.6.376
v0.6.375
v0.6.374
v0.6.373
v0.6.372
v0.6.371
v0.6.370
v0.6.369
v0.6.368
v0.6.367
v0.6.366
v0.6.365
v0.6.364
v0.6.363
v0.6.362
v0.6.361
v0.6.360
v0.6.359
v0.6.358
v0.6.357
v0.6.356
v0.6.355
v0.6.354
v0.6.353
v0.6.352
v0.6.351
v0.6.350
v0.6.349
v0.6.348
v0.6.347
v0.6.346
v0.6.345
v0.6.344
v0.6.343
v0.6.342
v0.6.341
v0.6.340
v0.6.339
v0.6.338
v0.6.337
v0.6.336
v0.6.335
v0.6.334
v0.6.333
v0.6.332
v0.6.331
v0.6.330
v0.6.329
v0.6.328
v0.6.327
v0.6.326
v0.6.325
v0.6.324
v0.6.323
v0.6.322
v0.6.321
v0.6.320
v0.6.319
v0.6.318
v0.6.317
v0.6.316
v0.6.315
v0.6.314
v0.6.313
v0.6.312
v0.6.311
v0.6.310
v0.6.309
v0.6.308
v0.6.307
v0.6.306
v0.6.305
v0.6.304
v0.6.303
v0.6.302
v0.6.301
v0.6.300
v0.6.299
v0.6.298
v0.6.297
v0.6.296
v0.6.295
v0.6.294
v0.6.293
v0.6.292
v0.6.291
v0.6.290
v0.6.289
v0.6.288
v0.6.287
v0.6.286
v0.6.285
v0.6.284
v0.6.283
v0.6.282
v0.6.281
v0.6.280
v0.6.279
v0.6.278
v0.6.277
v0.6.276
v0.6.275
v0.6.274
v0.6.273
v0.6.272
v0.6.271
v0.6.270
v0.6.269
v0.6.268
v0.6.267
v0.6.266
v0.6.265
v0.6.264
v0.6.263
v0.6.262
v0.6.261
v0.6.260
v0.6.259
v0.6.258
v0.6.257
v0.6.256
v0.6.255
v0.6.254
v0.6.253
v0.6.252
v0.6.251
v0.6.250
v0.6.249
v0.6.248
v0.6.247
v0.6.246
v0.6.245
v0.6.244
v0.6.243
v0.6.242
v0.6.241
v0.6.240
v0.6.239
v0.6.238
v0.6.237
v0.6.236
v0.6.235
v0.6.234
v0.6.233
v0.6.232
v0.6.231
v0.6.230
v0.6.229
v0.6.228
v0.6.227
v0.6.226
v0.6.225
v0.6.224
v0.6.223
v0.6.222
v0.6.221
v0.6.220
v0.6.219
v0.6.218
v0.6.217
v0.6.216
v0.6.215
v0.6.214
v0.6.213
v0.6.212
v0.6.211
v0.6.210
v0.6.209
v0.6.208
v0.6.207
v0.6.206
v0.6.205
v0.6.204
v0.6.203
v0.6.202
v0.6.201
v0.6.200
v0.6.199
v0.6.198
v0.6.197
v0.6.196
v0.6.195
v0.6.194
v0.6.193
v0.6.192
v0.6.191
v0.6.190
v0.6.189
v0.6.188
v0.6.187
v0.6.186
v0.6.185
v0.6.184
v0.6.183
v0.6.182
v0.6.181
v0.6.180
v0.6.179
v0.6.178
v0.6.177
v0.6.176
v0.6.175
v0.6.174
v0.6.173
v0.6.172
v0.6.171
v0.6.170
v0.6.169
v0.6.168
v0.6.167
v0.6.166
v0.6.165
v0.6.164
v0.6.163
v0.6.162
v0.6.161
v0.6.160
v0.6.159
v0.6.158
v0.6.157
v0.6.156
v0.6.155
v0.6.154
v0.6.153
v0.6.152
v0.6.151
v0.6.150
v0.6.149
v0.6.148
v0.6.147
v0.6.146
v0.6.145
v0.6.144
v0.6.143
v0.6.142
v0.6.141
v0.6.140
v0.6.139
v0.6.138
v0.6.137
v0.6.136
v0.6.135
v0.6.134
v0.6.133
v0.6.132
v0.6.131
v0.6.130
v0.6.129
v0.6.128
v0.6.127
v0.6.126
v0.6.125
v0.6.124
v0.6.123
v0.6.122
v0.6.121
v0.6.120
v0.6.119
v0.6.118
v0.6.117
v0.6.116
v0.6.115
v0.6.114
v0.6.113
v0.6.112
v0.6.111
v0.6.110
v0.6.109
v0.6.108
v0.6.107
v0.6.106
v0.6.105
v0.6.104
v0.6.103
v0.6.102
v0.6.101
v0.6.100
v0.6.99
v0.6.98
v0.6.97
v0.6.96
v0.6.95
v0.6.94
v0.6.93
v0.6.92
v0.6.91
v0.6.90
v0.6.89
v0.6.88
v0.6.87
v0.6.86
v0.6.85
v0.6.84
v0.6.83
v0.6.82
v0.6.81
v0.6.80
v0.6.79
v0.6.78
v0.6.77
v0.6.76
v0.6.75
v0.6.74
v0.6.73
v0.6.72
v0.6.71
v0.6.70
v0.6.69
v0.6.68
v0.6.67
v0.6.66
v0.6.65
v0.6.64
v0.6.63
v0.6.62
v0.6.61
v0.6.60
v0.6.59
v0.6.58
v0.6.57
v0.6.56
v0.6.55
v0.6.54
v0.6.53
v0.6.52
v0.6.51
v0.6.50
v0.6.49
v0.6.48
v0.6.47
v0.6.46
v0.6.45
v0.6.44
v0.6.43
v0.6.42
v0.6.41
v0.6.40
v0.6.39
v0.6.38
v0.6.37
v0.6.36
v0.6.35
v0.6.34
v0.6.33
v0.6.32
v0.6.31
v0.6.30
v0.6.29
v0.6.28
v0.6.25
v0.6.24
v0.6.23
v0.6.22
v0.6.21
v0.6.20
v0.6.19
v0.6.18
v0.6.17
v0.6.16
v0.6.15
v0.6.14
v0.6.12
v0.6.11
v0.6.10
v0.6.9
v0.6.8
v0.6.7
v0.6.6
v0.6.5
v0.6.4
v0.6.3
v0.6.2
v0.6.1
v0.6.0
v0.5.7
v0.5.6
v0.5.5
v0.5.4
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.12
v0.4.11
v0.4.10
v0.4.9
v0.4.8
v0.4.7
v0.4.6
v0.4.5
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.30
v0.3.29
v0.3.28
v0.3.27
v0.3.26
v0.3.25
v0.3.24
v0.3.23
v0.3.22
v0.3.21
v0.3.20
v0.3.19
v0.3.18
v0.3.17
v0.3.16
v0.3.15
v0.3.14
v0.3.13
v0.3.12
v0.3.11
v0.3.10
v0.3.9
v0.3.8
v0.3.7
v0.3.6
v0.3.5
v0.3.4
v0.3.3
v0.3.2
v0.3.1
v0.3.0
v0.2.7
v0.2.6
v0.2.5
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.0.3
v0.0.2
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference: Cordy/Cairn#123
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Surfaced while wiring #102 into
cmd/cairnd. The transport is merged (PR #122)but nothing constructs it yet, because this question has no answer in the code.
The problem
IMPLEMENTED-PEERING.md§4.3 is the core of Manuel's design: "the receivinginstance verifies the recipient exists locally before any bytes flow."
PrepareTransferneeds apeering.Recipientsimplementation to call.There isn't one, and it is not an oversight in the peering package — the
information does not exist:
localmode —auth.Localholdsusers map[string]string(unexported).Answerable, but needs an exported accessor.
oidcmode — the instance holds no list of users at all. A user doesnot exist locally until they first log in. Cairn consumes identity, it does
not implement it (ARCHITECTURE §5.4), so there is nothing to enumerate.
ldapmode — same:search-then-bindauthenticates a presentedcredential; it is not a roster.
So in two of the three auth modes there is currently nothing to ask.
Options
A. Home directory on the storage backend (recommended).
Define existence as "
/home/<recipient>exists". Uniform across all three authmodes, needs no new auth surface, and fails safe: a user who has never logged in
has no home, so the transfer is refused — which is correct, because delivery
would have had nowhere to put the file. Requires
auth.perUserHomes, whichpeering delivery already assumes (§5: delivery lands in
/home/<recipient>/Inbox/<peer>/).Cost: a peer can probe which users have ever logged in. §4.3 already accepts
that enumeration for authenticated, allow-listed peers.
B. Ask the IdP. Reuse #96's admin client to query the directory. Most
"correct" answer, but only works when the admin client is configured, adds a
network round trip inside
PrepareTransfer, and makes an inbound transferdepend on IdP availability.
C. Admin-maintained inbound allow-list per peering. Symmetric with the
outbound sender allow-list (§4.2), leaks nothing, and is fully deterministic —
but it is manual work per recipient and will be forgotten in practice.
D. Accept anything, decide at delivery. Rejected: it discards the property
§4.3 exists for and would let a peer upload gigabytes before discovering the
recipient is unknown.
Recommendation
A, with C available later as an optional narrowing for admins who do not
want the probe surface. If A is chosen,
auth.perUserHomesbecomes a hardprerequisite for enabling peering, and that belongs in the docs handbook and in
the admin UI at peering setup.
Also needed once this is decided
peering.*config: staging dir, external base URL (for the absolute uploadURL),
maxFileBytes.grpc.Server, registerpeering.Service, wraprootwithpeering.Mux, mountBlobHandlerat/peer/v1/blob/.Deliverer— that is #104.Natural companion to #103, which needs the same config surface.
Carried over from #118, which is closed for the part it was titled for (the dependency claims in README + ARCHITECTURE §1.4, settled in PR #122).
Its final paragraph also asked to fold in
example.config.json+ the deployment handbook page gaining thepeeringblock, and a handbook peering page (#109 pipeline). That deliberately did not ship with #122: beyondpeering.statePaththose config fields do not exist yet — they are blocked on this issue — and documenting them now would describe software that does not exist, which is the exact failure mode #118 was created to prevent.So it belongs here: whichever option is chosen, the config surface,
example.config.json, the handbook deployment page and the handbook peering page all land with the wiring.If option A is chosen, the handbook must also state plainly that
auth.perUserHomesis a hard prerequisite for enabling peering, and the admin UI should say so at peering setup rather than failing at the first inbound transfer.Decided (Nikola, 2026-08-05): A + C together.
Rationale: peering was defined from the start as admins choosing which users and groups may communicate with which peer. The per-recipient allow-list is manual by design, not a burden to be avoided — so there is no reason to ship A alone and treat C as a later narrowing.
A structural point this surfaces
The registry today has only the outbound direction.
Peer.Senders{Users,Groups}(§4.2) answers "who on THIS instance may send to that peer", andCanSendenforces it.C is the inbound mirror: "which of MY users may receive from that peer". That field does not exist. So A+C means extending
Peer:Same
Sendersstruct type, same fail-closed semantics, same live-group evaluation (#96) so revoking a group membership revokes the ability to receive within the cache TTL. The admin UI grows one more picker beside the existing one, which matches how an admin already thinks about a peering: these people, that peer, both directions.How A and C compose
AND, not OR. A recipient must be allow-listed for that peering and have a home. Fail closed, consistent with
CanSend.Corollary worth deciding deliberately: the refusal for "not allow-listed" and "no such user" should be the same
STATUS_UNKNOWN_RECIPIENT. §4.3 accepts enumeration for authenticated peers, but distinguishing the two would additionally leak "this user exists here, you just aren't cleared for them" — which is a fact about our org chart, not about the transfer. Losing that distinction costs a peer admin very little; they can ask.Consequences to honour when wiring
auth.perUserHomesbecomes a hard prerequisite. Peering setup must refuse at configuration time with a clear reason, not accept a transfer and fail at delivery. Note the dogfood instance may not currently have it enabled — verify before #106.IMPLEMENTED-PEERING.md§4.3 needs rewriting: it currently describes the existence check alone. It should describe A+C, the AND, and the single refusal code.Receiversfield, aCanReceivemirroringCanSend, both fail-closed on an unknown peer or empty list.Registry half landed (option C):
Peer.Receivers+CanReceive, 8 tests, fail-closed on unknown peer / empty username / empty list.Allowlistadded as a type alias forSenders, so stored JSON and existing callers are untouched.One thing implementing it surfaced
CanSendtakes a*auth.Userbecause the sender is authenticated locally — we have their session, so we have their live groups (#96).The recipient is not authenticated. They are a username on the wire, with no session to read groups from. So
CanReceivecannot resolve groups itself, and takes them as a parameter:The consequence, which needs to reach the admin UI and the handbook:
receiverslists work everywhere, in all three auth modes.receiverslists only work where the IdP admin client is configured (Keycloak admin client, #96). Without it the caller passesniland a group-only list matches nobody — correctly fail-closed, but silently useless.So an admin who sets a group-only inbound allow-list on an instance without the admin client has built something that refuses every transfer and gives no clue why. The admin UI should either refuse to save a group-only
receiverslist in that configuration, or warn plainly at save time. Cheaper to decide now than to debug later from a peer's side.Still open on this issue: the
cmd/cairndwiring (config fields,Recipientsimplementation combining residency +CanReceive, gRPC server construction,Mux, blob route).Decided (Nikola, 2026-08-05): live group evaluation, no materialization. Revisitable in v0.6 beta.
The question raised
Should assigning a group to a
receivers(orsenders) list cause Cairn to read that group's members from the IdP and store them as users — tracking whether each user was added via group or directly, so that:That provenance model is correct — set union with source attribution is exactly what a materialized design needs, and it names the three bugs a naive implementation would ship. Preserved here in full, because if we ever materialize, this is the design.
Why we are not doing it now
All three bugs are artefacts of materializing. Under live evaluation none exist: two groups is just "is any listed group in their current set", removing a group is deleting a string, and direct grants live in a separate list that group removal never touches. The bookkeeping only becomes necessary once membership has been copied out of the IdP.
It conflicts with a stated principle. §4.2 of this document says live evaluation "keeps identity consumed, never implemented — consistent with §5.4 of ARCHITECTURE.md", and the README leads with the same line. A stored, Cairn-maintained membership list is a small user directory, and keeping it true needs a sync loop — background job, state, failure modes — in a product that advertises no cron sidecar.
It reopens the revocation window. #96 shrank revocation from session lifetime to a 60s cache TTL; that was the point of the work. A materialized list moves it back to "until someone re-syncs", i.e. until an admin notices. Concretely: continuing to receive files after HR removed you from
records-unit.It makes the config illegible. The admin types
records-unit; materializing writes 40 usernames that will be wrong next week — and the provenance flags are then needed to reconstruct what the admin meant, which storing the intent never loses. The config should record the decision, not the computation.What we do instead
The pattern #96 already established — live resolution with a last-known-good fallback, availability over freshness:
This does not fix group-based lists on an instance with no IdP admin client (see the comment above). The honest position is that group lists are a feature of instances that gave Cairn a way to ask about groups; user-based lists work everywhere. The admin UI should say so rather than us building a directory to hide it.
The condition for changing our mind
If peering ever needs to work with the IdP offline for long stretches, materialization becomes the pragmatic answer and the provenance design above is the right one. Not a v0.5 requirement.
Flagged for v0.6 beta review — real usage between instances is the test of whether live resolution is fast enough and whether the no-admin-client limitation bites in practice.
Implemented — closing
Option A + C as decided, shipped across v0.4.1 and v0.4.2. This also unblocked the
cmd/cairndwiring that the issue title flagged.Registry: the inbound mirror
Peer.ReceiversalongsideSenders, withCanReceivemirroringCanSend— fail closed on an unknown peer, an empty username, or an empty list.Allowlistwas added as a type alias forSenders, so stored JSON and every existing caller are untouched.The test that matters is
TestSendPermissionDoesNotGrantReceivePermission: being cleared to send to a peer must not imply being cleared to receive from it. Those are different privileges and an admin granting one has not decided the other.The A+C composition
peerRecipients.Exists(peer, username)incmd/cairndANDs the two halves:CanReceive(peer, username, nil)/home/<username>exists on the backendBoth fail closed. The refusal is a single
STATUS_UNKNOWN_RECIPIENTfor either cause: Q4 already accepts that an authenticated peer can enumerate recipients, but distinguishing "no such user" from "not cleared for you" would additionally leak a fact about our org chart rather than about the transfer.The signature change this forced
Recipients.Existsoriginally took only a username. That cannot express A+C — the check is per-peering, so it needs to know which peering is asking. Changed toExists(peer, username string), with a test asserting the authenticated peer actually reaches the check so the rule cannot be silently reduced back to a global existence test.Known limitation, worth carrying forward
CanReceivetakes groups as a parameter rather than resolving them, because unlike a sender the recipient is not authenticated — they are a name on the wire with no session to read live groups from. The wiring passesnil.Consequence: user-based
receiverslists work in every auth mode; group-based lists need the IdP admin client. Without it a group-only list matches nobody — correctly fail-closed, but silently useless. The admin UI should refuse to save that combination, or warn plainly at save time, rather than letting an admin discover it from the peer's side. Carrying that to #105.Also settled here
auth.perUserHomesis a hard prerequisite for peering, since residency is the existence test and §5 delivers to/home/<recipient>/Inbox/<peer>/. It now defaults ON (v0.4.1), anddocs/handbook/deployment.mddocuments that changing it on a populated instance is a migration, not a toggle.