v0.1: Request logging, Prometheus metrics, graceful shutdown #4
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The server currently logs one startup line and nothing else. For dogfooding on the monitored cluster (Prometheus + Grafana + Alertmanager already in place):
slog/metricsPrometheus endpoint (stdlib-friendly: hand-rolled counters/histograms or the official client lib — decide and note the dependency trade-off in ARCHITECTURE.md)http.Server+ signal handling) — currentlyListenAndServewith no drain, which will show up as 502s during rolling updates on k3sThen a
homelab-alertsrule for CairnDown can join the existing alert groups.Implemented and pushed to main:
Metrics —
internal/obs/metrics.go: hand-rolled Prometheus text-exposition registry (stdlib only, dependency count stays at one; rationale in the package comment). Exposed unauthenticated at/metrics(aggregate counters only, no file/user data). Metrics:cairn_http_requests_total{route,method,code},cairn_http_request_duration_secondshistogram per route, request/response byte totals,cairn_build_info,cairn_start_time_seconds. Route label is a bounded class (api/dav/ui/health/metrics/share — never raw paths) and unknown HTTP verbs clamp toOTHER, so cardinality can't be inflated by hostile clients.Request logging —
internal/obs/middleware.go: outermost middleware, one slog line per request (method, path, status, dur_ms, bytes in/out, user, remote). Sees auth 401s;/healthzprobes are counted in metrics but excluded from logs (probe noise). Username is recovered from the inner auth layer via a context-recorder bridge ininternal/auth/context.go.Graceful shutdown —
http.Server+ SIGTERM/SIGINT handling, 15s bounded drain,ReadHeaderTimeoutset. k8s rollouts no longer surface as connection resets.Bonus fix (found by the new logging!) — first request logged
dur_ms=785: HTTP Basic re-sends credentials every request, so every request paid full argon2 cost. Added a memory-only verified-credentials cache (SHA-256 fingerprint, only populated after argon2 accepts; wrong-password and unknown-user timings stay equally slow, so no new timing oracle). Verified: first request ~850ms, subsequent 0ms.ServiceMonitor added to the staged homelab-config PR (
k3s/cairn/servicemonitor.yaml) so Prometheus scrapes Cairn the moment it's dogfooded — includes a note on checking theserviceMonitorSelectorlabel if metrics don't appear.Remaining from this issue: a
CairnDownalert rule inhomelab-alertsonce the dogfood deployment is live (needs the scrape working first).Tests: obs route-classing, render, middleware capture; auth fast-path cache. Full suite green; smoke-tested metrics output, log lines, and SIGTERM drain end to end.