v0.6.411: #742 batch B part 1, safety first (duplicate member guard, export end, keys follow the cluster, leases, census fixes) #1186

Merged
Cordy merged 71 commits from b10v07 into main 2026-11-19 20:49:40 +00:00
Owner

#742 batch B part 1, "safety first". Research, decisions, the approved mockup, the spec and plan, reviews and run logs are in .superpowers/sdd/2026-11-18-b10v07/.

Cluster safety

  • #1183 duplicate member names. A second process with a member's name is refused before join:

    • a route probe reads the peers' announced names before the embedded server starts;
    • NATS's own duplicate refusal counts as a refusal;
    • the member-record check stays.

    After join, the newer process stops without a left marker. The older one shows the state duplicate, the bell item cluster-member-duplicate and an audit line.

  • #1185 and the 5-member export. The exporting member acts on its own write. Every member acknowledges with its left marker and keeps NATS up until all have acknowledged (2 s grace, 60 s deadline). An export is refused while a member is not running.

  • P9 in-record leases. coord.Claim stops a holder at Until by its own clock, also while the store hangs and when it was paused. A stale copy-record run can no longer overwrite a newer one. Job leases follow the same rule (coord.Keeper).

  • Keys (#1178).

    • File content, link secrets, upload pieces and the drain fallback seal with the cluster's decided keys; the drain answers 409 keys-not-decided until the cluster converges.
    • The key ledger records which members hold each key.
    • New: cairnd cluster forget, the audit verbs key-drain-forced and key-drained, and a back-out WARN.
    • Refused: the remote crypto role with several instances, and members with different instanceSeeds.
  • Census of per-instance state. The bell, Activity, the readiness list, last sign-ins and space cosmetics follow other members. The trash purge, scheduled reports and device maintenance run once per cluster under leases. External mode logs disconnect and reconnect and re-checks with retries. Requests during an outage answer 503 coord-unavailable.

  • cairnd migrate --to nats skips converted v1 documents and deletes nothing.

Fixed, single instance too

  • Every data export with more than one collector failed at the ZIP step since #1023.
  • The outage watch dropped refusals made right after the last leader contact.
  • With instanceSeed, files sealed before a deployment-identity rotation opened only through the recovery identity.
  • The owner's End share no longer races an administrator's end.

UI. Duplicate chip and bell item, 4 new audit labels, the Re-encrypt 409 text, a State column in external mode, and the Instances section is rebuilt only on change. The mockup was approved, with 10 strings in EN/DE/FR/IT.

Docs. CHANGELOG v0.6.411, Known limitations rewritten, R13 "Forget a lost instance", corrections across the handbook, chart appVersion v0.6.411.

Verification

  • 4 waves, 5 fix rounds, and a final review with adversarial probes: 0 Critical or High open.
  • All Go packages pass, including 55 real-main multi-process tests; the web suite (323 files) and the a11y suite (93 files) pass.
  • VM run on 3 hosts with rc1 and rc2 (run-s13.md):
    • duplicate refused in 3 to 157 ms;
    • export of 3 members in about 6 s, of 5 in about 5 s;
    • forget works;
    • reconnect: rc2 showed no degraded spell in 381 polls.
  • Not run on the VMs: the data-export takeover under kill -STOP, the key back-out, and the bell spot check. Each is covered by deterministic real-main tests.
#742 batch B part 1, "safety first". Research, decisions, the approved mockup, the spec and plan, reviews and run logs are in `.superpowers/sdd/2026-11-18-b10v07/`. **Cluster safety** - **#1183 duplicate member names.** A second process with a member's name is refused before join: - a route probe reads the peers' announced names before the embedded server starts; - NATS's own duplicate refusal counts as a refusal; - the member-record check stays. After join, the newer process stops without a left marker. The older one shows the state `duplicate`, the bell item `cluster-member-duplicate` and an audit line. - **#1185 and the 5-member export.** The exporting member acts on its own write. Every member acknowledges with its left marker and keeps NATS up until all have acknowledged (2 s grace, 60 s deadline). An export is refused while a member is not running. - **P9 in-record leases.** `coord.Claim` stops a holder at Until by its own clock, also while the store hangs and when it was paused. A stale copy-record run can no longer overwrite a newer one. Job leases follow the same rule (`coord.Keeper`). - **Keys (#1178).** - File content, link secrets, upload pieces and the drain fallback seal with the cluster's decided keys; the drain answers 409 `keys-not-decided` until the cluster converges. - The key ledger records which members hold each key. - New: `cairnd cluster forget`, the audit verbs `key-drain-forced` and `key-drained`, and a back-out WARN. - Refused: the remote crypto role with several instances, and members with different `instanceSeed`s. - **Census of per-instance state.** The bell, Activity, the readiness list, last sign-ins and space cosmetics follow other members. The trash purge, scheduled reports and device maintenance run once per cluster under leases. External mode logs disconnect and reconnect and re-checks with retries. Requests during an outage answer 503 `coord-unavailable`. - `cairnd migrate --to nats` skips converted v1 documents and deletes nothing. **Fixed, single instance too** - Every data export with more than one collector failed at the ZIP step since #1023. - The outage watch dropped refusals made right after the last leader contact. - With `instanceSeed`, files sealed before a deployment-identity rotation opened only through the recovery identity. - The owner's End share no longer races an administrator's end. **UI.** Duplicate chip and bell item, 4 new audit labels, the Re-encrypt 409 text, a State column in external mode, and the Instances section is rebuilt only on change. The mockup was approved, with 10 strings in EN/DE/FR/IT. **Docs.** CHANGELOG v0.6.411, Known limitations rewritten, R13 "Forget a lost instance", corrections across the handbook, chart appVersion v0.6.411. **Verification** - 4 waves, 5 fix rounds, and a final review with adversarial probes: 0 Critical or High open. - All Go packages pass, including 55 real-main multi-process tests; the web suite (323 files) and the a11y suite (93 files) pass. - VM run on 3 hosts with rc1 and rc2 (`run-s13.md`): - duplicate refused in 3 to 157 ms; - export of 3 members in about 6 s, of 5 in about 5 s; - `forget` works; - reconnect: rc2 showed no degraded spell in 381 polls. - Not run on the VMs: the data-export takeover under `kill -STOP`, the key back-out, and the bell spot check. Each is covered by deterministic real-main tests.
The newer process missed the older run whenever its clock ran ahead by
more than the gap between the two beats. The read behind the landing CAS
attempt already proves a write by another run after our previous beat.
Also widens the hook test's 900 ms bound to a 5 s condition (L5).
After a pause the expired timer and the holder's work wake together in
either order. The work no longer passes a check that only looked at the
timer's channel: past Until, Held ends the claim and answers false.
TestAtRestKeyDerivation742a flaked: WaitMeta sees the leader's name
before the leader-elect finished its leader change, so a single create
went unanswered (30 s deadline) or answered 10005. Test-only fix.
# Conflicts:
#	cmd/cairnd/coordstart742a.go
# Conflicts:
#	cmd/cairnd/adminhandler742a.go
Cordy merged commit cf31bffa28 into main 2026-11-19 20:49:40 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#1186
No description provided.