Cordy/Cairn

Fork 0

Storage connect from the WebUI: bootstrap layer, setup mode, and the wizard #160

New issue

Closed

opened 2026-08-09 03:40:54 +00:00 by Cordy · 1 comment

Cordy commented

2026-08-09 03:40:54 +00:00

Owner

Decided with Nikola 2026-08-09 (raised while reviewing the v0.6 scaffold): operators may deploy Cairn without configuring storage and connect an S3 bucket or an already-mounted share from the admin panel. Ships fully in v0.6 so the v0.7 beta tests the first-run experience end to end.

Three-layer model (amends #152's config/state line):

Config (operator-written, file + env, read-only): listen, auth/IdP, encryption identities, optionally storage. Env wins.
Bootstrap (machine-written, local, new): exactly one thing — the storage connection, iff established via UI. Single file /data/.cairn/bootstrap.json, plaintext 0600 (same secret class and same trust level as the config sitting beside it; encrypting it to identities stored in that same config adds ritual, not security). Precedence: env > config > bootstrap > nothing.
State (the nine stores): with the data, per #152/#153.

The line, final form: the pointer to your data lives locally; everything it points to lives with the data. Declarative instances stay fully stateless pods; UI-connected instances need one small local volume for the pointer — the personas map exactly onto the mechanisms.

Setup mode:

Explicit sentinel storage.driver: "setup" — the zero-config default (posix /data) is untouched; absence never means setup.
In setup mode the embedded UI serves the storage wizard; every other API answers 503. Wizard access: config-provided admins when auth is configured; else the one-time log-printed token (existing mechanism).
Wizard options, honestly labeled: S3 (endpoint, bucket, keys, region) and Directory (posix path already mounted into the container — the UI states plainly that mounting NFS is the host's job; Cairn never mounts anything).
Apply: validate with a real write/read/delete probe against the target → persist bootstrap → graceful self-restart ("storage connected — restarting"); the restart policy brings the process up through the one normal boot path, storage-first (#153 ordering). No in-process hot-swap: a second initialization path is where the bugs live.
User creation (first-run /setup) comes AFTER storage connect in the flow — local users are state and need somewhere to live.

After connection: the panel shows the connection read-only (driver, endpoint/path, bucket, encryption mode) — the long-discussed storage card lands here. Changing the backend on a populated instance remains out of scope (supervised migration tool, v0.8+): it is a data migration wearing a settings costume.

Depends on #152 (statestore + boot order must accommodate setup mode) and #153. i18n en/de/fr/it for the wizard.

Decided with Nikola 2026-08-09 (raised while reviewing the v0.6 scaffold): operators may deploy Cairn **without configuring storage** and connect an S3 bucket or an already-mounted share from the admin panel. Ships fully in v0.6 so the v0.7 beta tests the first-run experience end to end. **Three-layer model (amends #152's config/state line):** 1. **Config** (operator-written, file + env, read-only): listen, auth/IdP, encryption identities, optionally storage. Env wins. 2. **Bootstrap** (machine-written, local, new): exactly one thing — the storage connection, iff established via UI. Single file `/data/.cairn/bootstrap.json`, plaintext `0600` (same secret class and same trust level as the config sitting beside it; encrypting it to identities stored in that same config adds ritual, not security). Precedence: **env > config > bootstrap > nothing**. 3. **State** (the nine stores): with the data, per #152/#153. The line, final form: *the pointer to your data lives locally; everything it points to lives with the data.* Declarative instances stay fully stateless pods; UI-connected instances need one small local volume for the pointer — the personas map exactly onto the mechanisms. **Setup mode:** - Explicit sentinel `storage.driver: "setup"` — the zero-config default (posix `/data`) is untouched; absence never means setup. - In setup mode the embedded UI serves the storage wizard; every other API answers 503. Wizard access: config-provided admins when auth is configured; else the one-time log-printed token (existing mechanism). - Wizard options, honestly labeled: **S3** (endpoint, bucket, keys, region) and **Directory** (posix path *already mounted into the container* — the UI states plainly that mounting NFS is the host's job; Cairn never mounts anything). - Apply: validate with a real write/read/delete probe against the target → persist bootstrap → **graceful self-restart** ("storage connected — restarting"); the restart policy brings the process up through the one normal boot path, storage-first (#153 ordering). No in-process hot-swap: a second initialization path is where the bugs live. - User creation (first-run `/setup`) comes AFTER storage connect in the flow — local users are state and need somewhere to live. **After connection:** the panel shows the connection **read-only** (driver, endpoint/path, bucket, encryption mode) — the long-discussed storage card lands here. Changing the backend on a populated instance remains out of scope (supervised migration tool, v0.8+): it is a data migration wearing a settings costume. Depends on #152 (statestore + boot order must accommodate setup mode) and #153. i18n en/de/fr/it for the wizard.

Cordy added this to the v0.6 — Stateless instance (state-in-backend) milestone

2026-08-09 03:40:54 +00:00

Cordy referenced this issue

2026-08-09 03:41:14 +00:00

v0.6 ADR: state store abstraction — envelope, atomicity, and the config/state line (gates the milestone) #152

Cordy referenced this issue

2026-08-09 03:41:24 +00:00

State-in-backend: instance state lives encrypted in the storage backend, not on local disk #138

Cordy referenced this issue from a commit

2026-08-09 06:35:34 +00:00

feat(#160): bootstrap layer - machine-written storage pointer (D11)

Cordy referenced this issue from a commit

2026-08-09 06:35:54 +00:00

test(#160): bootstrap - roundtrip, 0600, apply semantics, env precedence

Cordy referenced this issue from a commit

2026-08-09 06:36:32 +00:00

feat(#160): setup mode - wizard-only server, token gate, real probe, restart-into-normal-boot

Cordy referenced this issue from a commit

2026-08-09 06:36:46 +00:00

test(#160): probeStorage - posix roundtrip, validation

Cordy referenced this issue from a commit

2026-08-09 06:37:47 +00:00

feat(#160): storage-connect wizard page (en/de/fr/it, honest mount copy, live probe)

forgejo-actions referenced this issue from a commit

2026-08-09 06:48:09 +00:00

feat: #155 reserved-prefix guard, #156 instance lock, #160 setup mode + storage card (splices)

Cordy referenced this issue

2026-08-09 06:54:56 +00:00

v0.6 session 3: reserved-prefix guard (#155), instance lock (#156), storage connect + setup mode (#160) #165

Cordy referenced this issue from a commit

2026-08-09 06:55:16 +00:00

Merge pull request 'v0.6 session 3: reserved-prefix guard (#155), instance lock (#156), storage connect + setup mode (#160)' (#165) from feat-s3-guards into main

Cordy commented

2026-08-09 07:02:59 +00:00

Author

Owner

Shipped in v0.5.5 (PR #165). All three pieces landed per D11:

Bootstrap layer — internal/config/bootstrap.go: /data/.cairn/bootstrap.json (0600, 0700 dir, temp+rename; CAIRN_BOOTSTRAP_PATH override), holding exactly the storage connection and never encryption/quota/trash. The "setup" sentinel delegates the whole connection to the pointer, then storage env overrides are re-applied — env > config > bootstrap, tested (TestReapplyStorageEnvWins). Absence of a driver still means the zero-config posix default; only the explicit sentinel enters this path.

Setup mode — cmd/cairnd/setupmode.go: with the sentinel and no pointer, the process serves ONLY the wizard; / redirects, every other path answers 503 with an honest label; /healthz stays up for probes. Apply = decode → constant-time token check → real write/read/delete probe with content verification ("connected" means writable, not pingable) → persist pointer → respond → graceful shutdown → os.Exit(0), restarting through the one normal boot path. No hot swap, as ratified.

Wizard — web/static/storage-setup.html, same first-run visual identity as /setup, full en/de/fr/it. Two honestly-labeled options: S3, and Directory with the mounting truth stated in the option itself ("Mounting NFS or disks is the host's job — Cairn never mounts anything"). After apply it polls and hands over to the app when the restarted server answers.

Storage card — admin Settings now opens with a read-only card (driver, endpoint/bucket or path, encryption mode, and whether the connection came from the config file or this panel), backed by extended /api/v1/admin/info (no credentials in the payload). Backend change on a populated instance stays out of scope per the issue — the card's own hint says why.

One recorded deviation from D11: wizard access is the one-time log-printed token only; the "config-provided admins when auth is configured" path is not implemented in this cut. Building the full auth stack inside setup mode is real surface area for a persona (declarative IdP config + undeclared storage) that barely exists — an operator declaring OIDC declares a bucket. If the v0.7 beta surfaces that persona, this reopens.

Acceptance note: unit tests cover probe and bootstrap; the end-to-end first-run flow (fresh instance, sentinel, wizard, restart, /setup user creation after storage) will be exercised live as part of the #159 delete-the-volume rehearsal session, where a throwaway setup-mode instance costs nothing extra.

Shipped in **v0.5.5** (PR #165). All three pieces landed per D11: **Bootstrap layer** — `internal/config/bootstrap.go`: `/data/.cairn/bootstrap.json` (0600, 0700 dir, temp+rename; `CAIRN_BOOTSTRAP_PATH` override), holding exactly the storage connection and never encryption/quota/trash. The `"setup"` sentinel delegates the *whole* connection to the pointer, then storage env overrides are re-applied — env > config > bootstrap, tested (`TestReapplyStorageEnvWins`). Absence of a driver still means the zero-config posix default; only the explicit sentinel enters this path. **Setup mode** — `cmd/cairnd/setupmode.go`: with the sentinel and no pointer, the process serves ONLY the wizard; `/` redirects, every other path answers 503 with an honest label; `/healthz` stays up for probes. Apply = decode → constant-time token check → **real write/read/delete probe with content verification** ("connected" means writable, not pingable) → persist pointer → respond → graceful shutdown → `os.Exit(0)`, restarting through the one normal boot path. No hot swap, as ratified. **Wizard** — `web/static/storage-setup.html`, same first-run visual identity as `/setup`, full en/de/fr/it. Two honestly-labeled options: S3, and Directory with the mounting truth stated in the option itself ("Mounting NFS or disks is the host's job — Cairn never mounts anything"). After apply it polls and hands over to the app when the restarted server answers. **Storage card** — admin Settings now opens with a read-only card (driver, endpoint/bucket or path, encryption mode, and whether the connection came from the config file or this panel), backed by extended `/api/v1/admin/info` (no credentials in the payload). Backend change on a populated instance stays out of scope per the issue — the card's own hint says why. **One recorded deviation from D11:** wizard access is the one-time log-printed token only; the "config-provided admins when auth is configured" path is not implemented in this cut. Building the full auth stack inside setup mode is real surface area for a persona (declarative IdP config + undeclared storage) that barely exists — an operator declaring OIDC declares a bucket. If the v0.7 beta surfaces that persona, this reopens. **Acceptance note:** unit tests cover probe and bootstrap; the end-to-end first-run flow (fresh instance, sentinel, wizard, restart, `/setup` user creation after storage) will be exercised live as part of the #159 delete-the-volume rehearsal session, where a throwaway setup-mode instance costs nothing extra.

Cordy closed this issue

2026-08-09 07:03:07 +00:00

Cordy referenced this issue

2026-08-09 23:31:59 +00:00

Storage wizard: Basic-auth popup after restart handover + no language selector #166

Cordy referenced this issue

2026-08-09 23:36:13 +00:00

fix(#166): wizard restart handover without browser auth popup + language selector #167

Cordy referenced this issue

2026-08-09 23:51:40 +00:00

v0.6 acceptance: the delete-the-volume dogfood #159

Cordy referenced this issue