Port the nine state stores onto statestore + invert boot ordering #153
Labels
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Cordy/Cairn#153
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Depends on #152 (interface + implementations decided).
The nine: settings, holds, shares, app-passwords, local-users, peering registry, peer-transfers, license-state, notify-seen (#147). Each currently does its own
os.ReadFile/ temp+rename; the port swaps that forstatestore.Load/Saveand changes nothing else — same JSON, same in-memory structs, same locking. Names in the store are the existing file basenames, so alocalStorepointed at/data/.cairnreads today's files unchanged (zero-migration for posix instances).Boot ordering inversion (
main.go). Today settings/license/local-users load before the storage driver is built; with backendStore the driver must come first. Map the new order: config → storage driver (+ encryption) → statestore → settings → auth (local-users) → license → the rest. Define the failure mode "backend unreachable at boot": refuse to start (crash-loop until storage answers) rather than serve degraded — a file server without its storage is not degraded, it is down, and k8s/systemd restart semantics handle the retry. Readiness probe already covers rollout safety.Write-through semantics. Reads stay in-memory as today. Writes (admin actions, watermarks) go through
Savesynchronously; a failed Save fails the mutating request — no fire-and-forget, especially for holds. Watch the one hot-ish writer: notify-seen mark-all (fine) and peering transfers state updates during sends (low rate, but confirm).Acceptance: full test suite green with both implementations; a posix instance behaves byte-identically before/after; an s3 instance boots with an empty bucket and creates its state under
.cairn-state/as ciphertext.Shipped across v0.5.2–v0.5.4 (PRs #162, #163, #164), live on the dogfood.
Implementation deviated from the issue's sketch in one good way: no store structs or constructors changed.
statestoregained a path-binding shim (Bind/Read/Write) — main binds each configured state path to the encryptedBackendat boot on s3, before any store opens; each of the nine stores swapped exactly one read line and one write block forstatestore.Read/Write. Unbound paths behave byte-identically to pre-#153 (local temp-then-rename), so every existing test passed unchanged and posix instances are untouched. No boot reordering was needed either — the raw driver already precedes every store construction; the bind-and-migrate block slots directly after driver creation, below all decorators (content encryption never double-wraps state). D3 enforced: s3 without encryption identities refuses to boot. Write-through synchronous per D6.The recorded breadcrumb became two real deploy failures, worth the record:
storage.ErrNotFound, notfs.ErrNotExist— the exact thing #152 flagged for verification, verified the hard way. First fix joined the sentinel at themapS3Errsite (v0.5.3).Stathas a second, barereturn storage.ErrNotFoundfallthrough that bypassesmapS3Err, andOpenroutes throughStat. Systemic fix (v0.5.4): the sentinel itself now wraps —ErrNotFound = fmt.Errorf("not found: %w", fs.ErrNotExist)— so every return site, present and future, satisfies both checks. Lesson for the record: when two error vocabularies must agree, fix the definition, not the call sites.Both failures were clean refusals with zero partial state — the #154 semantics doing their job before #154 was even the one refusing.
Verified on the dogfood: v0.5.4 boots Running 1/1;
instance peering … enabled=truein the boot log provessettings.jsoncompleted the round trip — migrated into the Garage bucket, read back, decrypted.