Port the nine state stores onto statestore + invert boot ordering #153

Closed
opened 2026-08-09 03:16:40 +00:00 by Cordy · 1 comment
Owner

Depends on #152 (interface + implementations decided).

The nine: settings, holds, shares, app-passwords, local-users, peering registry, peer-transfers, license-state, notify-seen (#147). Each currently does its own os.ReadFile / temp+rename; the port swaps that for statestore.Load/Save and changes nothing else — same JSON, same in-memory structs, same locking. Names in the store are the existing file basenames, so a localStore pointed at /data/.cairn reads today's files unchanged (zero-migration for posix instances).

Boot ordering inversion (main.go). Today settings/license/local-users load before the storage driver is built; with backendStore the driver must come first. Map the new order: config → storage driver (+ encryption) → statestore → settings → auth (local-users) → license → the rest. Define the failure mode "backend unreachable at boot": refuse to start (crash-loop until storage answers) rather than serve degraded — a file server without its storage is not degraded, it is down, and k8s/systemd restart semantics handle the retry. Readiness probe already covers rollout safety.

Write-through semantics. Reads stay in-memory as today. Writes (admin actions, watermarks) go through Save synchronously; a failed Save fails the mutating request — no fire-and-forget, especially for holds. Watch the one hot-ish writer: notify-seen mark-all (fine) and peering transfers state updates during sends (low rate, but confirm).

Acceptance: full test suite green with both implementations; a posix instance behaves byte-identically before/after; an s3 instance boots with an empty bucket and creates its state under .cairn-state/ as ciphertext.

Depends on #152 (interface + implementations decided). **The nine:** settings, holds, shares, app-passwords, local-users, peering registry, peer-transfers, license-state, notify-seen (#147). Each currently does its own `os.ReadFile` / temp+rename; the port swaps that for `statestore.Load/Save` and changes nothing else — same JSON, same in-memory structs, same locking. Names in the store are the existing file basenames, so a `localStore` pointed at `/data/.cairn` reads today's files unchanged (zero-migration for posix instances). **Boot ordering inversion (`main.go`).** Today settings/license/local-users load before the storage driver is built; with backendStore the driver must come first. Map the new order: config → storage driver (+ encryption) → statestore → settings → auth (local-users) → license → the rest. Define the failure mode "backend unreachable at boot": **refuse to start** (crash-loop until storage answers) rather than serve degraded — a file server without its storage is not degraded, it is down, and k8s/systemd restart semantics handle the retry. Readiness probe already covers rollout safety. **Write-through semantics.** Reads stay in-memory as today. Writes (admin actions, watermarks) go through `Save` synchronously; a failed Save fails the mutating request — no fire-and-forget, especially for holds. Watch the one hot-ish writer: notify-seen mark-all (fine) and peering transfers state updates during sends (low rate, but confirm). Acceptance: full test suite green with both implementations; a posix instance behaves byte-identically before/after; an s3 instance boots with an empty bucket and creates its state under `.cairn-state/` as ciphertext.
Cordy closed this issue 2026-08-09 04:16:56 +00:00
Author
Owner

Shipped across v0.5.2–v0.5.4 (PRs #162, #163, #164), live on the dogfood.

Implementation deviated from the issue's sketch in one good way: no store structs or constructors changed. statestore gained a path-binding shim (Bind/Read/Write) — main binds each configured state path to the encrypted Backend at boot on s3, before any store opens; each of the nine stores swapped exactly one read line and one write block for statestore.Read/Write. Unbound paths behave byte-identically to pre-#153 (local temp-then-rename), so every existing test passed unchanged and posix instances are untouched. No boot reordering was needed either — the raw driver already precedes every store construction; the bind-and-migrate block slots directly after driver creation, below all decorators (content encryption never double-wraps state). D3 enforced: s3 without encryption identities refuses to boot. Write-through synchronous per D6.

The recorded breadcrumb became two real deploy failures, worth the record:

  1. v0.5.2 crashlooped: the s3 driver's 404 maps to storage.ErrNotFound, not fs.ErrNotExist — the exact thing #152 flagged for verification, verified the hard way. First fix joined the sentinel at the mapS3Err site (v0.5.3).
  2. v0.5.3 still crashlooped: Stat has a second, bare return storage.ErrNotFound fallthrough that bypasses mapS3Err, and Open routes through Stat. Systemic fix (v0.5.4): the sentinel itself now wraps — ErrNotFound = fmt.Errorf("not found: %w", fs.ErrNotExist) — so every return site, present and future, satisfies both checks. Lesson for the record: when two error vocabularies must agree, fix the definition, not the call sites.

Both failures were clean refusals with zero partial state — the #154 semantics doing their job before #154 was even the one refusing.

Verified on the dogfood: v0.5.4 boots Running 1/1; instance peering … enabled=true in the boot log proves settings.json completed the round trip — migrated into the Garage bucket, read back, decrypted.

Shipped across **v0.5.2–v0.5.4** (PRs #162, #163, #164), live on the dogfood. **Implementation** deviated from the issue's sketch in one good way: no store structs or constructors changed. `statestore` gained a **path-binding shim** (`Bind/Read/Write`) — main binds each configured state path to the encrypted `Backend` at boot on s3, before any store opens; each of the nine stores swapped exactly one read line and one write block for `statestore.Read/Write`. Unbound paths behave byte-identically to pre-#153 (local temp-then-rename), so every existing test passed unchanged and posix instances are untouched. No boot reordering was needed either — the raw driver already precedes every store construction; the bind-and-migrate block slots directly after driver creation, below all decorators (content encryption never double-wraps state). D3 enforced: s3 without encryption identities refuses to boot. Write-through synchronous per D6. **The recorded breadcrumb became two real deploy failures, worth the record:** 1. v0.5.2 crashlooped: the s3 driver's 404 maps to `storage.ErrNotFound`, not `fs.ErrNotExist` — the exact thing #152 flagged for verification, verified the hard way. First fix joined the sentinel at the `mapS3Err` site (v0.5.3). 2. v0.5.3 **still** crashlooped: `Stat` has a second, bare `return storage.ErrNotFound` fallthrough that bypasses `mapS3Err`, and `Open` routes through `Stat`. Systemic fix (v0.5.4): the **sentinel itself** now wraps — `ErrNotFound = fmt.Errorf("not found: %w", fs.ErrNotExist)` — so every return site, present and future, satisfies both checks. Lesson for the record: when two error vocabularies must agree, fix the *definition*, not the call sites. Both failures were clean refusals with zero partial state — the #154 semantics doing their job before #154 was even the one refusing. **Verified on the dogfood:** v0.5.4 boots Running 1/1; `instance peering … enabled=true` in the boot log proves `settings.json` completed the round trip — migrated into the Garage bucket, read back, decrypted.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#153
No description provided.