Check-in logging and refusal history (#40)
Shipped 2026-09-06 (PR #41), deployed via deploy.yml. The server-side half of the 2026-08 licence-drop incident; the client half is Cairn #437 (v0.6.133, the 14-day revoked watch).
Why this exists
In August 2026 cairn-enc's healthy 60-seat licence dropped to the free tier and stayed there for days. Post-incident, the server could tell us nothing: the check-in handler logged no successes, no refusals, no reasons, and lastSeen was a single overwritten value. The most likely cause — a 403 with reason unknown answered while the server was mid-redeploy during the JSON→SQLite migration — could only be inferred, never proven. This change makes the next incident reconstructable from the server alone, and removes the most dangerous refusal entirely.
What is logged now
Every /v1/checkin outcome leaves a journal line:
level=INFO msg="checkin ok" id=lic-… org=… seats=… expires=… keyReturned=true
level=WARN msg="checkin refused" id=lic-… reason=revoked|expired|unknown refusals=N
level=ERROR msg="checkin: store empty or unavailable - answering 503, not 403-unknown" id=lic-…
level=WARN msg="checkin bad request" remote=…
Read them with:
sudo journalctl -u cairn-license-server --no-pager | grep checkin
Refusal history on the record
Every 403 for a known licence stamps the record, persisted in the SQLite data JSON:
| field | meaning |
|---|---|
lastRefusedAt |
UTC time of the most recent refused check-in |
lastRefusedReason |
revoked or expired (an unknown id has no record to stamp — the journal covers those) |
refusalCount |
total refusals over the record's lifetime |
The history is a trail, not state: a later successful check-in does not clear it. Read it on the VPS:
sudo sqlite3 /var/lib/cairn-license/licenses.db \
"SELECT id,
json_extract(data,'$.lastRefusedAt'),
json_extract(data,'$.lastRefusedReason'),
json_extract(data,'$.refusalCount'),
json_extract(data,'$.lastSeen')
FROM licenses;"
503, not 403-unknown, when the store cannot answer
A check-in that misses the in-memory records is only answered 403 unknown when the store is healthy (DB reachable and at least one licence loaded). An empty or unreachable store — the mid-migration position that refused the incident licence — answers 503 instead. Cairn clients treat 503 like connectivity failure: daily retry on the grace path, no self-revocation, fully self-healing. A production licence server holding zero licences is a migration accident, not a clean slate.
With real licences loaded, an unknown id remains an honest 403 reason=unknown (pinned by TestUnknownIDStillRefusedWhenStoreHealthy).
Interplay with the Cairn client (#437)
Even when a bad 403 does slip out, clients on ≥ v0.6.133 no longer treat it as terminal: revocation is a 14-day watch with daily check-ins, so one wrong answer heals at the next 200. Defence in depth: the server tries never to give the wrong answer (503 guard), and the client survives it if one gets through (revoked watch).
Incident playbook
journalctl … | grep "checkin refused"— who was refused, when, why, how often.- The SQL above — per-licence refusal history plus
lastSeen, surviving restarts. - On the instance side, the Cairn licence page names the state (refused / unreachable / expired, with dates), the admin bell keeps drop/restore transitions, and
cairn_license_licensed==0withinstalled==1fires theCairnLicenseDroppedalert in the homelab.