Cordy/Cairn

Fork 0

Live UI updates via SSE: /api/v1/events nudge stream (first consumer: Federated requests) #476

New issue

Closed

opened 2026-09-10 11:14:40 +00:00 by Cordy · 1 comment

Cordy commented

2026-09-10 11:14:40 +00:00

Owner

Dogfood finding: an inbound OCM share does not appear in Federated → Requests until a page refresh. The bell self-heals within ~1–2 min (60s pollBell + 45s server notification cache), but the open view never re-fetches — office users read that as "broken". Decision (Nikola, 2026-09-10): build real push via SSE, not a poll piggyback.

Why SSE, not WebSocket

Traffic is strictly server→client ("something changed"), so EventSource fits exactly: plain HTTP (no upgrade handling at Traefik), automatic reconnection with backoff built into the browser, no client library, no new dependency server-side. WebSocket buys bidirectionality we do not need and costs upgrade/proxy complexity.

Design: nudges, not payloads

The stream never carries data — only topic nudges: event: change / data: {"topic":"ocm"}. The client reacts by re-fetching the existing REST endpoints it already trusts. This keeps the derived-view architecture (the bell derives, nothing is stored) and means the stream needs no serialisation, no i18n, and no per-item authorisation of its own — the REST layer keeps that job. Nudges are idempotent, so dropping one under pressure is harmless (the poll fallback catches it).

Topics for v1: ocm (inbound share arrived / decided elsewhere) and bell (any notification-relevant change). Transfers, locks, licence can join later without protocol changes.

Backend

New internal/events: Hub with Subscribe(user) (<-chan string, cancel func()) and Publish(user, topic string). Per-user fanout map + mutex; small buffered channels; drop-on-full (never block a mutation path on a slow browser); per-user connection cap (~4) to bound resources.
Handler GET /api/v1/events (session-authenticated, registered like the other user APIs): headers Content-Type: text/event-stream, Cache-Control: no-cache, X-Accel-Buffering: no (harmless on Traefik, saves nginx users); cast to http.Flusher, flush after every frame; heartbeat comment line every ~25s so proxy idle timeouts never cut a quiet stream; exit on r.Context().Done() and always run cancel().
Write-timeout trap: if cairnd's http.Server sets WriteTimeout, a long-lived stream dies at that deadline. Use http.ResponseController.SetWriteDeadline to extend per heartbeat (or confirm WriteTimeout is unset). Check this FIRST — it is the classic silent SSE killer.
Publish call sites: ocm inbound shares handler after AddShareIn (nudge the recipient, topics ocm+bell); ocmShareInDecide (same user — another tab/device decided); peering inbox delivery; notifications-relevant licence transitions. Start with the OCM two; wire the rest as one-liners later.
Observability: gauge cairn_sse_connections; connect/disconnect at debug level.

Frontend

On login/boot: new EventSource("/api/v1/events"); on change events parse the topic — ocm: run pollBell() and, if the Federated view is the one on screen, fedReload(); bell: pollBell().
Keep the existing 60s pollBell interval as the fallback (SSE connected → it is just cheap belt-and-braces; consider stretching to 5 min while the stream is healthy).
EventSource reconnects automatically; after a redeploy (in-memory sessions) reconnection gets 401 → existing session-expiry handling takes over, polling still covers the gap. No custom reconnect code.
Insecure-origin note: EventSource works fine on plain http, unlike some clipboard APIs — no dogfood caveat.

Testing (house TDD)

Hub unit tests: fanout to two subscribers, unsubscribe cleanup, drop-on-full does not block, connection cap.
Handler httptest: subscribe → Publish → client reads a well-formed event:/data: frame; heartbeat appears; client disconnect frees the subscription (assert via hub count).
Integration: POST a wire share at /ocm/shares → subscribed recipient stream receives the ocm nudge.

Out of scope v1

Event history / Last-Event-ID replay — nudges make replay pointless (a reconnect just re-fetches).
Cross-pod fanout: dogfood runs single-replica Recreate, so an in-process hub is correct today. The state-in-backend future (#138) with multiple replicas would need a shared pub/sub — record the constraint, do not build it.

Docs

Handbook: short "live updates" note — proxy buffering (nginx X-Accel-Buffering), heartbeat interval, and that polling remains the fallback so SSE is an enhancement, never a requirement.

Dogfood finding: an inbound OCM share does not appear in Federated → Requests until a page refresh. The bell self-heals within ~1–2 min (60s `pollBell` + 45s server notification cache), but the open view never re-fetches — office users read that as "broken". Decision (Nikola, 2026-09-10): build real push via **SSE**, not a poll piggyback. ## Why SSE, not WebSocket Traffic is strictly server→client ("something changed"), so `EventSource` fits exactly: plain HTTP (no upgrade handling at Traefik), automatic reconnection with backoff built into the browser, no client library, no new dependency server-side. WebSocket buys bidirectionality we do not need and costs upgrade/proxy complexity. ## Design: nudges, not payloads The stream never carries data — only **topic nudges**: `event: change` / `data: {"topic":"ocm"}`. The client reacts by re-fetching the existing REST endpoints it already trusts. This keeps the derived-view architecture (the bell derives, nothing is stored) and means the stream needs no serialisation, no i18n, and no per-item authorisation of its own — the REST layer keeps that job. Nudges are idempotent, so dropping one under pressure is harmless (the poll fallback catches it). Topics for v1: `ocm` (inbound share arrived / decided elsewhere) and `bell` (any notification-relevant change). Transfers, locks, licence can join later without protocol changes. ## Backend - New `internal/events`: `Hub` with `Subscribe(user) (<-chan string, cancel func())` and `Publish(user, topic string)`. Per-user fanout map + mutex; small buffered channels; **drop-on-full** (never block a mutation path on a slow browser); per-user connection cap (~4) to bound resources. - Handler `GET /api/v1/events` (session-authenticated, registered like the other user APIs): headers `Content-Type: text/event-stream`, `Cache-Control: no-cache`, `X-Accel-Buffering: no` (harmless on Traefik, saves nginx users); cast to `http.Flusher`, flush after every frame; **heartbeat comment line every ~25s** so proxy idle timeouts never cut a quiet stream; exit on `r.Context().Done()` and always run cancel(). - **Write-timeout trap:** if `cairnd`'s `http.Server` sets `WriteTimeout`, a long-lived stream dies at that deadline. Use `http.ResponseController.SetWriteDeadline` to extend per heartbeat (or confirm WriteTimeout is unset). Check this FIRST — it is the classic silent SSE killer. - Publish call sites: ocm inbound `shares` handler after `AddShareIn` (nudge the recipient, topics `ocm`+`bell`); `ocmShareInDecide` (same user — another tab/device decided); peering inbox delivery; notifications-relevant licence transitions. Start with the OCM two; wire the rest as one-liners later. - Observability: gauge `cairn_sse_connections`; connect/disconnect at debug level. ## Frontend - On login/boot: `new EventSource("/api/v1/events")`; on `change` events parse the topic — `ocm`: run `pollBell()` and, if the Federated view is the one on screen, `fedReload()`; `bell`: `pollBell()`. - Keep the existing 60s `pollBell` interval as the fallback (SSE connected → it is just cheap belt-and-braces; consider stretching to 5 min while the stream is healthy). - `EventSource` reconnects automatically; after a redeploy (in-memory sessions) reconnection gets 401 → existing session-expiry handling takes over, polling still covers the gap. No custom reconnect code. - Insecure-origin note: `EventSource` works fine on plain http, unlike some clipboard APIs — no dogfood caveat. ## Testing (house TDD) - Hub unit tests: fanout to two subscribers, unsubscribe cleanup, drop-on-full does not block, connection cap. - Handler httptest: subscribe → Publish → client reads a well-formed `event:`/`data:` frame; heartbeat appears; client disconnect frees the subscription (assert via hub count). - Integration: POST a wire share at `/ocm/shares` → subscribed recipient stream receives the `ocm` nudge. ## Out of scope v1 - Event history / `Last-Event-ID` replay — nudges make replay pointless (a reconnect just re-fetches). - Cross-pod fanout: dogfood runs single-replica `Recreate`, so an in-process hub is correct today. The state-in-backend future (#138) with multiple replicas would need a shared pub/sub — record the constraint, do not build it. ## Docs Handbook: short "live updates" note — proxy buffering (nginx `X-Accel-Buffering`), heartbeat interval, and that polling remains the fallback so SSE is an enhancement, never a requirement.

Cordy added this to the v0.7 — Closed beta (dogfood + hardening) milestone

2026-09-10 14:56:33 +00:00

Cordy referenced this issue from a commit

2026-09-10 15:00:13 +00:00

scout: main.go wiring regions for #476 (never commits, delete after)

Cordy referenced this issue from a commit

2026-09-10 15:06:13 +00:00

one-shot: SSE nudge stream, red-then-green (#476)

forgejo-actions referenced this issue from a commit

2026-09-10 15:06:50 +00:00

events: SSE nudge stream, Federated and the bell update live (#476)

Cordy referenced this issue from a pull request that will close it,

2026-09-10 15:09:28 +00:00

events: SSE nudge stream, Federated and the bell update live (#476) #477

Cordy referenced this issue from a commit

2026-09-10 15:11:40 +00:00

Merge pull request 'events: SSE nudge stream, Federated and the bell update live (#476)' (#477) from fix-476 into main

Cordy closed this issue

2026-09-10 15:11:40 +00:00

Cordy commented

2026-09-10 15:21:03 +00:00

Author

Owner

Shipped in v0.6.157, live on both dogfoods (cairn_build_info{version="v0.6.157"} verified; /api/v1/events answers 401 unauthenticated through the ingress, so the route is registered and reachable).

Built exactly per the brief, house TDD (stubs first, red witnessed on the runner, then implementation): internal/events.Hub (fanout, drop-on-full, per-user cap 4) + session-authenticated SSE at GET /api/v1/events (nudge frames only, 25s heartbeats, X-Accel-Buffering: no), publish on inbound share arrival and on decide/dismiss, startEvents() in the frontend with the 60s poll kept as fallback. PR #477. The WriteTimeout trap flagged in this issue turned out moot — the main server deliberately sets none (its own comment says so, for large transfers).

Deferred, recorded on the PR: cairn_sse_connections gauge, additional publishers (peering inbox, licence transitions), ocm-package integration test for the inbound nudge.

Dogfood verification (needs a signed-in session, so over to Nikola): fresh login on files-bao, open Federated → Requests, then create an OCM share from Nextcloud towards nikola-test — the request card and the bell dot should appear within a second or two, no refresh. A second browser/tab on the Shares view should also update when you accept in the first.

Shipped in **v0.6.157**, live on both dogfoods (`cairn_build_info{version="v0.6.157"}` verified; `/api/v1/events` answers 401 unauthenticated through the ingress, so the route is registered and reachable). Built exactly per the brief, house TDD (stubs first, red witnessed on the runner, then implementation): `internal/events.Hub` (fanout, drop-on-full, per-user cap 4) + session-authenticated SSE at `GET /api/v1/events` (nudge frames only, 25s heartbeats, `X-Accel-Buffering: no`), publish on inbound share arrival and on decide/dismiss, `startEvents()` in the frontend with the 60s poll kept as fallback. PR #477. The WriteTimeout trap flagged in this issue turned out moot — the main server deliberately sets none (its own comment says so, for large transfers). Deferred, recorded on the PR: `cairn_sse_connections` gauge, additional publishers (peering inbox, licence transitions), ocm-package integration test for the inbound nudge. **Dogfood verification** (needs a signed-in session, so over to Nikola): fresh login on files-bao, open Federated → Requests, then create an OCM share from Nextcloud towards nikola-test — the request card and the bell dot should appear within a second or two, no refresh. A second browser/tab on the Shares view should also update when you accept in the first.

Cordy referenced this issue

2026-09-10 22:25:22 +00:00

Change-feed endpoint: GET /api/v1/changes (unblocks near-instant desktop sync) #478

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#476

No description provided.

Rows
Columns