Check-ins are invisible: log every one, persist last refusal per licence (Cairn #437 companion) #40

Closed
opened 2026-09-06 01:55:35 +00:00 by Cordy · 1 comment
Owner

During the cairn-enc 60→50 incident (Cairn #437) the server side could not answer a single question: the /v1/checkin handler logs nothing — not successes, not 403s, not reasons — and lastSeen is one overwritten timestamp. A customer's instance self-revoked off a 403 this server almost certainly emitted during the Aug 24–26 redeploy/migration era, and there is no trace.

Build:

  1. slog every check-in: licence id, outcome (ok / refused), refusal reason (revoked / expired / unknown), and whether a refreshed key was returned. One line per request; this endpoint is monthly-per-instance, volume is trivial.
  2. Persist refusal history per licence: last_refused_at, last_refused_reason, refusal_count on the licence record (staff console surfacing can follow later).
  3. Bonus alignment: with Cairn #437's client-side revoked-watch, a transient unknown here self-heals within a day — but this server should strive never to 403 unknown for a licence it merely can't read yet (e.g. mid-migration): consider 503 for storage-unavailable, which the client already treats as connectivity (grace path) rather than revocation.
During the cairn-enc 60→50 incident (Cairn #437) the server side could not answer a single question: the `/v1/checkin` handler logs nothing — not successes, not 403s, not reasons — and `lastSeen` is one overwritten timestamp. A customer's instance self-revoked off a 403 this server almost certainly emitted during the Aug 24–26 redeploy/migration era, and there is no trace. Build: 1. **slog every check-in**: licence id, outcome (ok / refused), refusal reason (revoked / expired / unknown), and whether a refreshed key was returned. One line per request; this endpoint is monthly-per-instance, volume is trivial. 2. **Persist refusal history per licence**: `last_refused_at`, `last_refused_reason`, `refusal_count` on the licence record (staff console surfacing can follow later). 3. Bonus alignment: with Cairn #437's client-side revoked-watch, a transient `unknown` here self-heals within a day — but this server should strive never to 403 `unknown` for a licence it merely can't read *yet* (e.g. mid-migration): consider 503 for storage-unavailable, which the client already treats as connectivity (grace path) rather than revocation.
Cordy closed this issue 2026-09-06 03:16:21 +00:00
Author
Owner

Shipped in PR #41 (merged), deployed to the VPS via deploy.yml (healthz ok, service active).

What landed:

  • Every check-in logs: checkin ok (id/org/seats/expires/keyReturned), checkin refused (id/reason/refusals), bad-request warns.
  • Refusal history persisted on the record: lastRefusedAt / lastRefusedReason / refusalCount in the SQLite data JSON — stamped per 403 for known licences, never cleared by success, survives store reopen (tested).
  • 503 instead of 403-unknown when the store cannot answer honestly (empty or DB unreachable) — the mid-redeploy position that refused cairn-enc's healthy licence in August now tells clients "retry later" and their grace path handles it. Unknown ids on a healthy store stay honest 403s (pinned by test).

TDD: 503 + logging tests witnessed red on the runner (empty store answered 403; nothing logged); refusal-history fields witnessed compile-red; full suite green.

Live verify: probe check-in against production → 403 {"reason":"unknown"} and the journal now shows checkin refused id=lic-verify-40 reason=unknown refusals=0.

Docs: wiki page Check-in Logging and Refusal History (log lines, SQL to read the history, 503 semantics, incident playbook) + entry in the Cairn wiki Home working log.

Together with Cairn #437 (client-side 14-day revoked watch, v0.6.133) the incident class is closed from both ends.

Shipped in PR #41 (merged), deployed to the VPS via `deploy.yml` (healthz ok, service active). **What landed:** - Every check-in logs: `checkin ok` (id/org/seats/expires/keyReturned), `checkin refused` (id/reason/refusals), bad-request warns. - Refusal history persisted on the record: `lastRefusedAt` / `lastRefusedReason` / `refusalCount` in the SQLite `data` JSON — stamped per 403 for known licences, never cleared by success, survives store reopen (tested). - **503 instead of 403-unknown when the store cannot answer honestly** (empty or DB unreachable) — the mid-redeploy position that refused cairn-enc's healthy licence in August now tells clients "retry later" and their grace path handles it. Unknown ids on a healthy store stay honest 403s (pinned by test). **TDD:** 503 + logging tests witnessed red on the runner (empty store answered 403; nothing logged); refusal-history fields witnessed compile-red; full suite green. **Live verify:** probe check-in against production → `403 {"reason":"unknown"}` and the journal now shows `checkin refused id=lic-verify-40 reason=unknown refusals=0`. **Docs:** wiki page [Check-in Logging and Refusal History](http://192.168.10.245/Cordy/cairn-license-server/wiki/Check-in+Logging+and+Refusal+History.-) (log lines, SQL to read the history, 503 semantics, incident playbook) + entry in the Cairn wiki Home working log. Together with Cairn #437 (client-side 14-day revoked watch, v0.6.133) the incident class is closed from both ends.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/cairn-license-server#40
No description provided.