Versions panel and API show the encrypted size of a version, not the file size #947

Closed
opened 2026-10-17 02:01:03 +00:00 by Cordy · 4 comments
Owner

Found in the v0.6.347 local live check (2026-10-17). v0.6.346 does the same.

What happens: on an encrypted instance, GET /api/v1/versions reports each version's stored size, which is the age ciphertext. A 6-byte file shows size: 304. The Versions panel shows that number next to each version, so a small file looks about 50 times bigger than it is. It also doesn't match the size the file list shows for the current file.

Expected: the plaintext size, as the file list shows it.

To do:

  • Find where the file list gets its plaintext size (ownermeta manifest or encrypt-layer stat).
  • Report the same for versions: decrypt the header, or record the plaintext size in the record. Watch the cost for long chains.
  • Add a test on an encrypted rig: version size equals the plaintext length.
Found in the v0.6.347 local live check (2026-10-17). v0.6.346 does the same. **What happens:** on an encrypted instance, `GET /api/v1/versions` reports each version's stored size, which is the age ciphertext. A 6-byte file shows `size: 304`. The Versions panel shows that number next to each version, so a small file looks about 50 times bigger than it is. It also doesn't match the size the file list shows for the current file. **Expected:** the plaintext size, as the file list shows it. **To do:** - Find where the file list gets its plaintext size (ownermeta manifest or encrypt-layer stat). - Report the same for versions: decrypt the header, or record the plaintext size in the record. Watch the cost for long chains. - Add a test on an encrypted rig: version size equals the plaintext length.
Author
Owner

Deep research (2026-10-17)

The full write-up, with sources and a checked formula, is in ~/cairn-work/scratch/b7e/backlog/research-947.md.

Findings

  • The listed size alone isn't enough. How big the age header is depends on the recipients (168, 266, 1627 or 3184 bytes are all seen in practice). If Cairn guesses the header length wrong, it gets a number that age still accepts: a 6-byte file comes out as 2924 B.
  • Once the header length is known, the size is exact. The plaintext size is the stored size minus the header, minus 16, minus 16 per 64 KiB chunk. That is age's own internal formula, and it needs no key. I checked it against 28 out of 28 cases. The 304 B in the issue is exactly owner plus recovery key: 266 + 16 + 6 + 16.
  • What other products do:
    • Cryptomator, rclone crypt and gocryptfs use a fixed-size header, so they get the size from the listing alone.
    • Nextcloud, ownCloud, oCIS, Seafile and restic store the size next to the file.
  • The stored-size designs are where the bugs are. Nextcloud and ownCloud have shown 0 B, sizes 4/3 too large, truncated downloads, false "Bad Signature" errors, and broken version restores (nextcloud#63662, PR #64484, PR #49903, owncloud#22781). oCIS reports a revision as 0 bytes when its size field can't be read.
  • Cairn already avoids the worst of it. Version downloads take their length from the decrypted bytes, not the listed size.

Options

  • (b) Store the size with each version: still needs (a) underneath.
    • S3 listings don't return metadata, so it still costs a request per version.
    • S3 none mode keeps no metadata at all.
    • posix xattrs need NFSv4.2.
    • Old versions have no stored size.
  • (c) A manifest for versions: breaks VERSIONING §1 (no metadata index).
  • (a) Read each version's own header, cache the size in memory, store nothing new.
    • Cost on S3: one ranged GET of 8 KiB, the first time a file's versions are listed.
    • Cost on posix: one open and read.
    • Later listings cost nothing.

Recommendation

Option (a).

What users would see

The real size, matching the file list. Nothing changes on unencrypted instances, for quota, or for downloads.

Decision needed

  1. Derive the size from the header (a), rather than (b) or (c)?
  2. If a header can't be read, show the stored size for that row and try again next time?
  3. An empty version currently shows "298 B" and would show "anna · " with a dangling dot. Drop the dot when there's no size?
## Deep research (2026-10-17) The full write-up, with sources and a checked formula, is in `~/cairn-work/scratch/b7e/backlog/research-947.md`. ### Findings - **The listed size alone isn't enough.** How big the age header is depends on the recipients (168, 266, 1627 or 3184 bytes are all seen in practice). If Cairn guesses the header length wrong, it gets a number that age still accepts: a 6-byte file comes out as 2924 B. - **Once the header length is known, the size is exact.** The plaintext size is the stored size minus the header, minus 16, minus 16 per 64 KiB chunk. That is age's own internal formula, and it needs no key. I checked it against 28 out of 28 cases. The 304 B in the issue is exactly owner plus recovery key: 266 + 16 + 6 + 16. - **What other products do:** - Cryptomator, rclone crypt and gocryptfs use a fixed-size header, so they get the size from the listing alone. - Nextcloud, ownCloud, oCIS, Seafile and restic store the size next to the file. - **The stored-size designs are where the bugs are.** Nextcloud and ownCloud have shown 0 B, sizes 4/3 too large, truncated downloads, false "Bad Signature" errors, and broken version restores (nextcloud#63662, PR #64484, PR #49903, owncloud#22781). oCIS reports a revision as 0 bytes when its size field can't be read. - **Cairn already avoids the worst of it.** Version downloads take their length from the decrypted bytes, not the listed size. ### Options - **(b) Store the size with each version:** still needs (a) underneath. - S3 listings don't return metadata, so it still costs a request per version. - S3 none mode keeps no metadata at all. - posix xattrs need NFSv4.2. - Old versions have no stored size. - **(c) A manifest for versions:** breaks VERSIONING §1 (no metadata index). - **(a) Read each version's own header, cache the size in memory, store nothing new.** - Cost on S3: one ranged GET of 8 KiB, the first time a file's versions are listed. - Cost on posix: one open and read. - Later listings cost nothing. ### Recommendation Option (a). ### What users would see The real size, matching the file list. Nothing changes on unencrypted instances, for quota, or for downloads. ### Decision needed 1. Derive the size from the header (a), rather than (b) or (c)? 2. If a header can't be read, show the stored size for that row and try again next time? 3. An empty version currently shows "298 B" and would show "anna · " with a dangling dot. Drop the dot when there's no size?
Author
Owner

Decided by Nikola (2026-10-17):

  • Plain size: Cairn derives each version's plain size from its own age header, cached in memory. Nothing new is stored.
  • Unreadable header: that row shows the stored size, and Cairn tries again next time.
  • Empty versions: no dangling dot.

Implementation: batch 2.

**Decided by Nikola (2026-10-17):** - **Plain size:** Cairn derives each version's plain size from its own age header, cached in memory. Nothing new is stored. - **Unreadable header:** that row shows the stored size, and Cairn tries again next time. - **Empty versions:** no dangling dot. Implementation: batch 2.
Cordy closed this issue 2026-10-17 20:40:30 +00:00
Author
Owner

Shipped in v0.6.349 (PR #959). Both dogfoods run it.

On encrypted instances, version sizes are now the real file sizes. Each size is read from the version's own age header without a key, and kept in an in-memory cache of 4096 entries. Nothing extra is stored. An empty version no longer shows a trailing dot.

The live check (the Versions panel shows 6 B for a 6-byte file) comes after the next sign-in.

**Shipped in v0.6.349 (PR #959). Both dogfoods run it.** On encrypted instances, version sizes are now the real file sizes. Each size is read from the version's own age header without a key, and kept in an in-memory cache of 4096 entries. Nothing extra is stored. An empty version no longer shows a trailing dot. The live check (the Versions panel shows 6 B for a 6-byte file) comes after the next sign-in.
Author
Owner

Live result (both dogfoods, v0.6.349, 2026-10-18; recorded on PR #959, copied here): PASS. I saved a 6-byte file three times. Its versions now list as 6 bytes; before this change they showed the encrypted size, about 304.

**Live result (both dogfoods, v0.6.349, 2026-10-18; recorded on PR #959, copied here): PASS.** I saved a 6-byte file three times. Its versions now list as `6` bytes; before this change they showed the encrypted size, about 304.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#947
No description provided.