Legacy-key coverage report: find files still encrypted to the retained recovery key #348

Closed
opened 2026-08-28 00:21:40 +00:00 by Cordy · 3 comments
Owner

Why

Rotating the recovery identity is not retroactive. Recipients() adds the recovery key at write time and nothing revisits a stored object, so after a rotation every pre-rotation file opens only with the key now sitting in recoveryIdentityLegacy.

Cairn keeps exactly one legacy slot. That gives an operator a hard ceiling: they can rotate once, and cannot safely rotate again until no stored object depends on the retained key — because a second rotation evicts it and orphans anything still encrypted to it.

Today the operator has no way to find out whether that condition holds. docs/handbook/encryption.md ("Retiring an old recovery key") can only offer wait it out or rewrite everything and hope, and explicitly tells them to keep the old key indefinitely because they cannot prove otherwise. That is an honest answer to a question the product should simply answer.

This issue is the first version of that answer, and it is what unblocks repeat rotation as a supported operation.

Scope (v1)

An admin-triggered scan that reports which stored objects are still encrypted to the retained legacy recovery key.

  • Button on the Encryption & keys admin page, visible only when recoveryIdentityLegacy is set.
  • Walks the storage backend and, for each object, reads the age header only — age.ExtractHeader gives the stanzas without decrypting the payload, so this is a header scan, not a decrypt pass.
  • Classifies each object as: covered by the active recovery key / covered only by the legacy key / covered by neither (a real anomaly worth surfacing loudly).
  • Produces a report the operator can act on: counts per class, and the paths in the legacy-only class.
  • Long-running and resumable-ish by nature — it must not block the request. Run it as a background job with progress, and write the report where the operator can fetch it afterwards.

Definition of done

  • An operator who has rotated once can press one button and get a truthful answer to "can I delete the old key yet?"
  • When the legacy-only count reaches zero, the report says so plainly, and the handbook's retirement section can point at this instead of hedging.
  • The handbook's "Retiring an old recovery key" section is rewritten to use the feature.

Deliberately out of scope here

Automatic re-encryption of the legacy-only set, scheduling, and any multi-key generalisation. Those belong to the follow-up deep dive (see the companion issue on legacy-slot handling) — this issue only has to make the current state visible.

Notes for implementation

  • Header-only reads keep this cheap, but object listing on a large bucket is still the dominant cost; assume it is measured in minutes and design the UI around that.
  • Stanza flavour matters: mlkem768x25519 vs X25519 distinguishes post-quantum from classical objects and should appear in the report, since a PQ migration and a rotation can be in flight at the same time.
  • The same mechanism answers the equivalent question for deploymentIdentityLegacy; keep the scan generic over "retained identity" rather than hardcoding the recovery slot.
## Why Rotating the recovery identity is not retroactive. `Recipients()` adds the recovery key at write time and nothing revisits a stored object, so after a rotation every pre-rotation file opens **only** with the key now sitting in `recoveryIdentityLegacy`. Cairn keeps exactly one legacy slot. That gives an operator a hard ceiling: they can rotate once, and cannot safely rotate again until no stored object depends on the retained key — because a second rotation evicts it and orphans anything still encrypted to it. Today the operator has no way to find out whether that condition holds. `docs/handbook/encryption.md` ("Retiring an old recovery key") can only offer *wait it out* or *rewrite everything and hope*, and explicitly tells them to keep the old key indefinitely because they cannot prove otherwise. That is an honest answer to a question the product should simply answer. This issue is the first version of that answer, and it is what unblocks repeat rotation as a supported operation. ## Scope (v1) An admin-triggered scan that reports which stored objects are still encrypted to the retained legacy recovery key. - Button on the **Encryption & keys** admin page, visible only when `recoveryIdentityLegacy` is set. - Walks the storage backend and, for each object, reads the age header only — `age.ExtractHeader` gives the stanzas without decrypting the payload, so this is a header scan, not a decrypt pass. - Classifies each object as: covered by the **active** recovery key / covered **only by the legacy** key / covered by neither (a real anomaly worth surfacing loudly). - Produces a report the operator can act on: counts per class, and the paths in the legacy-only class. - Long-running and resumable-ish by nature — it must not block the request. Run it as a background job with progress, and write the report where the operator can fetch it afterwards. ## Definition of done - An operator who has rotated once can press one button and get a truthful answer to "can I delete the old key yet?" - When the legacy-only count reaches zero, the report says so plainly, and the handbook's retirement section can point at this instead of hedging. - The handbook's "Retiring an old recovery key" section is rewritten to use the feature. ## Deliberately out of scope here Automatic re-encryption of the legacy-only set, scheduling, and any multi-key generalisation. Those belong to the follow-up deep dive (see the companion issue on legacy-slot handling) — this issue only has to make the current state *visible*. ## Notes for implementation - Header-only reads keep this cheap, but object listing on a large bucket is still the dominant cost; assume it is measured in minutes and design the UI around that. - Stanza flavour matters: `mlkem768x25519` vs `X25519` distinguishes post-quantum from classical objects and should appear in the report, since a PQ migration and a rotation can be in flight at the same time. - The same mechanism answers the equivalent question for `deploymentIdentityLegacy`; keep the scan generic over "retained identity" rather than hardcoding the recovery slot.
Author
Owner

Design decided (2026-09-03, mockups in key-rotation-mockup.html): variant B — lifecycle stepper.

Encryption & keys gains a "Retired recovery key" card, present only while a *Legacy identity is configured, headed by the ADR 0002 lifecycle drawn as a four-step rail (Rotate ✓ → Re-encrypt → Verify → Discard) with live sub-labels. Card states: coverage unknown → scanning (background, resumable, "headers only · no decryption") → report (three count cells: active / retired-only / neither, with PQ-flavour split and paths grouped by prefix with counts — never thousands of raw lines) → green verdict naming the exact config field to remove. "Covered by neither" renders as a red cell with an always-open path list. The Re-encrypt button's home is reserved but ships with the drain follow-up, not here.

Implementation findings from the design pass:

  • X25519 stanzas deliberately don't identify their recipient, so classification cannot come from reading stanza args. Instead: trial-unwrap — hand the first ~64 KiB to age.Decrypt with one identity at a time; nil error = covered, ErrIncorrectIdentity = not. Public API only, microseconds per object per key, payload never read. The S3 driver's lazy ranged Open means this costs one small ranged GET per object; no new driver capability.
  • The scan must run on the bare base driver (below reserved and encrypt): coverage truth includes the .cairn-state/*.age objects — the recovery key seals those too, and a "covers nothing" verdict that ignored them would be a lie.
  • Scanner is generic over retained identities per this issue's note: classifies against {actives} / {retirings}, so deploymentIdentityLegacy uses the same machinery.
  • Server-side prefix aggregation for the report (the high-file-count constraint surfacing in the UI, exactly as the design-first pass intended).
  • v1 report lives in memory (rescan after a pod restart); the persistent drain ledger arrives with the drain per ADR 0002.

Building now: internal/keyscan + GET/POST /api/v1/admin/keyscan + the stepper/card UI + i18n ×4 + handbook touch.

**Design decided (2026-09-03, mockups in `key-rotation-mockup.html`): variant B — lifecycle stepper.** Encryption & keys gains a "Retired recovery key" card, present only while a `*Legacy` identity is configured, headed by the ADR 0002 lifecycle drawn as a four-step rail (**Rotate ✓ → Re-encrypt → Verify → Discard**) with live sub-labels. Card states: coverage unknown → scanning (background, resumable, "headers only · no decryption") → report (three count cells: active / retired-only / neither, with PQ-flavour split and paths **grouped by prefix with counts** — never thousands of raw lines) → green verdict naming the exact config field to remove. "Covered by neither" renders as a red cell with an always-open path list. The Re-encrypt button's home is reserved but ships with the drain follow-up, not here. **Implementation findings from the design pass:** - X25519 stanzas deliberately don't identify their recipient, so classification cannot come from reading stanza args. Instead: **trial-unwrap** — hand the first ~64 KiB to `age.Decrypt` with one identity at a time; nil error = covered, `ErrIncorrectIdentity` = not. Public API only, microseconds per object per key, payload never read. The S3 driver's lazy ranged `Open` means this costs one small ranged GET per object; no new driver capability. - The scan must run on the **bare base driver** (below `reserved` and `encrypt`): coverage truth includes the `.cairn-state/*.age` objects — the recovery key seals those too, and a "covers nothing" verdict that ignored them would be a lie. - Scanner is generic over retained identities per this issue's note: classifies against {actives} / {retirings}, so `deploymentIdentityLegacy` uses the same machinery. - Server-side prefix aggregation for the report (the high-file-count constraint surfacing in the UI, exactly as the design-first pass intended). - v1 report lives in memory (rescan after a pod restart); the persistent drain ledger arrives with the drain per ADR 0002. Building now: `internal/keyscan` + `GET/POST /api/v1/admin/keyscan` + the stepper/card UI + i18n ×4 + handbook touch.
Cordy referenced this issue from a commit 2026-09-03 17:52:10 +00:00
Author
Owner

Shipped — v0.6.115 (PR #409), live on both dogfoods.

Built exactly to the variant-B design: internal/keyscan trial-unwrap scanner over the bare base driver (state objects included), GET/POST/DELETE /api/v1/admin/keyscan, and the Rotation tab on Encryption & keys with the lifecycle stepper, report grid, prefix-grouped paths, and the green discard verdict. Handbook retirement section rewritten to point at the scan. Two red-cycle fixes on the branch (missing package-dir mkdir in the one-shot; my own wrong test expectations for the two-segment grouping rule — the code was right).

Dogfood note: both dogfoods have no recoveryIdentityLegacy, so the Rotation tab is correctly hidden today — that in itself is checklist item 1. To exercise the full flow without a real rotation:

  1. Encryption & keys shows only Custody/Recovery tabs (clean instance → no Rotation tab).
  2. Generate a throwaway key (age-keygen), put its secret into recoveryIdentityLegacy in the cairn-enc-config secret (untracked — Argo won't revert), restart the pod.
  3. Rotation tab appears → stepper shows Rotate ✓ / coverage unknown → Scan coverage → progress counts up (~1,000s of objects on the dogfood, headers only) → report should land at 0 retired-only (the throwaway key never covered anything) → green "Covers nothing" verdict, step 4 active, instructions naming recoveryIdentityLegacy.
  4. Remove the legacy entry, restart → tab disappears again.
  5. Languages spot-check on the stepper + verdict copy.

The amber still-covered state and the Re-encrypt button arrive with the drain follow-up (ADR 0002 rollout); a real rotation on the dogfood would light the amber path up today.

**Shipped — v0.6.115 (PR #409), live on both dogfoods.** Built exactly to the variant-B design: `internal/keyscan` trial-unwrap scanner over the bare base driver (state objects included), `GET/POST/DELETE /api/v1/admin/keyscan`, and the Rotation tab on Encryption & keys with the lifecycle stepper, report grid, prefix-grouped paths, and the green discard verdict. Handbook retirement section rewritten to point at the scan. Two red-cycle fixes on the branch (missing package-dir mkdir in the one-shot; my own wrong test expectations for the two-segment grouping rule — the code was right). **Dogfood note:** both dogfoods have no `recoveryIdentityLegacy`, so the Rotation tab is correctly *hidden* today — that in itself is checklist item 1. To exercise the full flow without a real rotation: 1. Encryption & keys shows only Custody/Recovery tabs (clean instance → no Rotation tab). 2. Generate a throwaway key (`age-keygen`), put its secret into `recoveryIdentityLegacy` in the `cairn-enc-config` secret (untracked — Argo won't revert), restart the pod. 3. Rotation tab appears → stepper shows Rotate ✓ / coverage unknown → **Scan coverage** → progress counts up (~1,000s of objects on the dogfood, headers only) → report should land at **0 retired-only** (the throwaway key never covered anything) → green "Covers nothing" verdict, step 4 active, instructions naming `recoveryIdentityLegacy`. 4. Remove the legacy entry, restart → tab disappears again. 5. Languages spot-check on the stepper + verdict copy. The amber still-covered state and the Re-encrypt button arrive with the drain follow-up (ADR 0002 rollout); a real rotation on the dogfood would light the amber path up today.
Author
Owner

Closing: shipped in v0.6.115 (scan + Rotation tab, variant B stepper) and polished in v0.6.116 (permanent tab with idle explainer, docs link, flavour chips, ago-fix, 409 guard). Operator pass complete on both dogfoods — idle state, live amber lifecycle after a real rotation on cairn-openbao, and the genuine classical-legacy coverage report on cairn-enc (24 active / 3 retired-only / 0 neither). Handbook rewritten to point at the feature.

The re-encryption that drives these counts to zero is #411.

Closing: shipped in v0.6.115 (scan + Rotation tab, variant B stepper) and polished in v0.6.116 (permanent tab with idle explainer, docs link, flavour chips, ago-fix, 409 guard). Operator pass complete on both dogfoods — idle state, live amber lifecycle after a real rotation on cairn-openbao, and the genuine classical-legacy coverage report on cairn-enc (24 active / 3 retired-only / 0 neither). Handbook rewritten to point at the feature. The re-encryption that drives these counts to zero is #411.
Cordy closed this issue 2026-09-03 19:20:51 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#348
No description provided.