State lives with the data again on s3: bind the ten unbound stores and pictures (#820) #822

Merged
Cordy merged 2 commits from fix-820 into main 2026-09-29 18:06:03 +00:00
Owner

Fixes #820. Awaiting review before merge.

What changes

  • statePaths moves out of main() into cmd/cairnd/statepaths.go and gains the eight JSON stores (spaceids, spacemeta, spacequota, spacealerts, tags, favorites, recents, activity). They already used statestore.Read/Write; only the binding was missing.
  • Profile pictures and space images go through a new statestore.Blobs (read cache incl. misses). On s3: Backend.Sub("avatars"|"spaceimg") → .cairn-state/avatars/<user>.jpg.age, .cairn-state/spaceimg/<id>.jpg.age. On posix: the same local dirs as before (byte-identical layout, no migration).
  • First boot on s3 runs MigrateDir (the #154 adopt-or-refuse rule per picture) and retires local copies to *.migrated.
  • ETags are now content hashes (no file mtime on the backend).
  • keydrain: state objects one level below the prefix are re-sealed via the matching Sub; before this, key rotation would have failed on every picture.
  • activity: saves batched to one per fixed 2s window (5000-event file ≈ 1 MB encrypted PUT per file op otherwise); flushed on shutdown.
  • Guard: TestEveryStateFileIsBound parses main.go; every filepath.Join(filepath.Dir(cfg.…), "<name>") must be in statePaths or localOnlyState. Verified red: removing the #820 entries lists all eight. It also caught marks (leak-tracing ledger), now declared local beside audit.
  • Docs: ARCHITECTURE §11, deployment.md "What still lives locally", chart comment, CHANGELOG v0.6.310.

Tests: go vet ./... and go test ./... green locally; new: statestore Sub/Blobs/MigrateDir, avatar via Blobs across a "restart", keydrain sub-prefix (verified red before fix), activity burst = 1 save, guard + basename uniqueness.

Fixes #820. Awaiting review before merge. **What changes** - `statePaths` moves out of `main()` into `cmd/cairnd/statepaths.go` and gains the eight JSON stores (spaceids, spacemeta, spacequota, spacealerts, tags, favorites, recents, activity). They already used `statestore.Read/Write`; only the binding was missing. - Profile pictures and space images go through a new `statestore.Blobs` (read cache incl. misses). On s3: `Backend.Sub("avatars"|"spaceimg")` → `.cairn-state/avatars/<user>.jpg.age`, `.cairn-state/spaceimg/<id>.jpg.age`. On posix: the same local dirs as before (byte-identical layout, no migration). - First boot on s3 runs `MigrateDir` (the #154 adopt-or-refuse rule per picture) and retires local copies to `*.migrated`. - ETags are now content hashes (no file mtime on the backend). - **keydrain**: state objects one level below the prefix are re-sealed via the matching `Sub`; before this, key rotation would have failed on every picture. - **activity**: saves batched to one per fixed 2s window (5000-event file ≈ 1 MB encrypted PUT per file op otherwise); flushed on shutdown. - **Guard**: `TestEveryStateFileIsBound` parses `main.go`; every `filepath.Join(filepath.Dir(cfg.…), "<name>")` must be in `statePaths` or `localOnlyState`. Verified red: removing the #820 entries lists all eight. It also caught `marks` (leak-tracing ledger), now declared local beside audit. - Docs: ARCHITECTURE §11, deployment.md "What still lives locally", chart comment, CHANGELOG v0.6.310. **Tests**: `go vet ./...` and `go test ./...` green locally; new: statestore Sub/Blobs/MigrateDir, avatar via Blobs across a "restart", keydrain sub-prefix (verified red before fix), activity burst = 1 save, guard + basename uniqueness.
State lives with the data again on s3: bind the ten unbound stores and pictures (#820)
All checks were successful
ci / test-and-build (pull_request) Successful in 1m30s
1862c07c4c
- statePaths moves to cmd/cairnd/statepaths.go and gains spaceids,
  spacemeta, spacequota, spacealerts, tags, favorites, recents, activity
- profile pictures + space images go through statestore.Blobs: Sub
  backends (.cairn-state/avatars|spaceimg) on s3, local dirs on posix;
  one-time MigrateDir with the #154 adopt-or-refuse rule
- keydrain re-seals sub-prefix state objects through the matching Sub
- activity saves batched to one per 2s window (whole-object PUTs on s3);
  flushed on shutdown
- guard test: every state location derived in main.go must be bound or
  declared in localOnlyState (marks + preview-cache)
- docs: ARCHITECTURE s11, deployment.md local table, chart comment,
  CHANGELOG v0.6.310
Blobs: a slow Get must not clobber a concurrent Put's cache entry (#820 review)
All checks were successful
ci / test-and-build (pull_request) Successful in 1m16s
bc76e3cba9
Cordy merged commit 365ab71776 into main 2026-09-29 18:06:03 +00:00
Cordy deleted branch fix-820 2026-09-29 18:06:07 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#822
No description provided.