Versions: after a move or copy onto a file with its own history, merged versions can age from the other file's dates #946

Closed
opened 2026-10-17 00:49:49 +00:00 by Cordy · 5 comments
Owner

Found in the fix-922 final review (2026-10-17).

Background

Since #922, a version's age runs from when it was replaced. The engine takes that moment from the next newer entry in the file's version chain.

What happens

Moving or copying file A onto file B merges A's chain into B's, ordered by time. The two histories then interleave, and an entry can take its "replaced at" time from the other file's entry.

Example

  • A holds a1, saved 300 days ago, which a2 replaced yesterday.
  • B holds b1, saved 250 days ago.
  • After moving A onto B, a1's next newer entry is b1. So a1 counts as replaced 250 days ago and the age cap (90 days by default) removes it at once. v0.6.346 kept it for 90 days.

This needs a move or copy onto a file that has its own older history.

Options

  • Store the replacement time per entry when the replacing save publishes, for example in object metadata or a sidecar.
  • Or age merged entries per origin.

To do

  • Choose an option.
  • Add a test for the scenario above on all backends.
  • Document the retention rule.
Found in the fix-922 final review (2026-10-17). ## Background Since #922, a version's age runs from when it was replaced. The engine takes that moment from the next newer entry in the file's version chain. ## What happens Moving or copying file A onto file B merges A's chain into B's, ordered by time. The two histories then interleave, and an entry can take its "replaced at" time from the other file's entry. ## Example - A holds a1, saved 300 days ago, which a2 replaced yesterday. - B holds b1, saved 250 days ago. - After moving A onto B, a1's next newer entry is b1. So a1 counts as replaced 250 days ago and the age cap (90 days by default) removes it at once. v0.6.346 kept it for 90 days. This needs a move or copy onto a file that has its own older history. ## Options - Store the replacement time per entry when the replacing save publishes, for example in object metadata or a sidecar. - Or age merged entries per origin. ## To do - Choose an option. - Add a test for the scenario above on all backends. - Document the retention rule.
Author
Owner

Related case from the second final re-review (2026-10-17), with the same root cause (merge re-keying):

  • Before: an older version kept from before #922, with a whole-second key, ages from its own key time. That time is when it was replaced.
  • What changes it: if a move onto another file clashes on its name, mergeKey re-keys the version through newKey (versions.go ~1470). The new key carries nanoseconds, so the version no longer counts as old.
  • Effect: it then ages from the next newer entry instead. That only keeps it longer, never shorter.

The per-entry replacement time proposed above would cover this case as well.

Related case from the second final re-review (2026-10-17), with the same root cause (merge re-keying): - **Before:** an older version kept from before #922, with a whole-second key, ages from its own key time. That time is when it was replaced. - **What changes it:** if a move onto another file clashes on its name, `mergeKey` re-keys the version through `newKey` (versions.go ~1470). The new key carries nanoseconds, so the version no longer counts as old. - **Effect:** it then ages from the next newer entry instead. That only keeps it longer, never shorter. The per-entry replacement time proposed above would cover this case as well.
Author
Owner

Deep research (2026-10-17)

The full write-up, with sources, a simulation and a test plan, is in ~/cairn-work/scratch/b7e/backlog/research-946.md.

What actually merges. Only one operation joins two histories: moving or renaming a file onto a path whose version folder already has entries (Rename calls mergeChain). Copies never merge.

Four ways a version expires too early. The issue's example is one of them. The other three:

  • B's version just below A's current record ages from A's current record.
  • B's replaced content ages from A's old last save.
  • A's current record later ages from B's newer versions.

In each case the version disappears from the list at once and is deleted at the next save.

What other products do. None of them mixes two histories.

  • Nextcloud and ownCloud 10 leave the moved file's versions behind, and those age from save time.
  • OpenCloud deletes the destination before the move.
  • Drive moves a file by changing its parents.
  • The products that count age from replacement store that time on the version itself: SharePoint's SnapshotDate, Syncthing's name tag or mtime, Seafile's commit cutoff.

What broke for others.

  • Syncthing, 2019: cleanout read the wrong timestamp and deleted a 2017 file on the day it was archived. The fix keeps the archive time in the version's name.
  • Nextcloud AIO users only discovered the 30-day pruning when they needed a restore.
  • restic and borg warn that mixing unrelated histories breaks retention.

Options rejected.

  • (b) S3 metadata, or posix mtime/xattr: fails in S3 put and none modes, needs a HEAD per entry on every listing, and is lost on moves.
  • (c) A sidecar per chain: a metadata index with a read-modify-write race.
  • (d) Aging per origin: still needs a key change and is less precise.
  • (e) Document only: leaves the silent losses in place.

Recommendation: (a). At merge time, write the replacement time into the version's key: <12 digits>r<unix>[a|u], 98 bytes at most.

  • Works on posix and every S3 mode, needs no index or leases, and survives backups.
  • v0.6.347 and v0.6.348 already read these names, so a rollback is safe.
  • The simple rule from batch-2 research is not enough. In a simulation of 20,000 random merges it still aged 12,061 entries too early.
  • The corrected algorithm pins A's versions for free and pins B's versions only where needed, before A's arrive. It produced 0 early expiries in 120,000 checked entries, at about 1.5 renames per merge.

What users would see. Every version of both files is kept for 90 days from when it was replaced. A few destination versions get new IDs, so a versions panel that was open during the move needs a refresh.

Side finding. VERSIONING §7 says Nextcloud drops history on copy. Current Nextcloud copies it, so the docs need fixing.

Decision needed

  1. Adopt (a), replacement time in the key at merge?
  2. Keep the 87-byte name guard? A version whose name would not fit stays unpinned and logs a WARN.
  3. Accept a one-hour tolerance? Uploads that write a temp file and move it over the original then cost no extra rename, at the price of a version expiring up to one hour early.
## Deep research (2026-10-17) The full write-up, with sources, a simulation and a test plan, is in `~/cairn-work/scratch/b7e/backlog/research-946.md`. **What actually merges.** Only one operation joins two histories: moving or renaming a file onto a path whose version folder already has entries (`Rename` calls `mergeChain`). Copies never merge. **Four ways a version expires too early.** The issue's example is one of them. The other three: - B's version just below A's current record ages from A's current record. - B's replaced content ages from A's old last save. - A's current record later ages from B's newer versions. In each case the version disappears from the list at once and is deleted at the next save. **What other products do.** None of them mixes two histories. - Nextcloud and ownCloud 10 leave the moved file's versions behind, and those age from save time. - OpenCloud deletes the destination before the move. - Drive moves a file by changing its parents. - The products that count age from replacement store that time on the version itself: SharePoint's `SnapshotDate`, Syncthing's name tag or mtime, Seafile's commit cutoff. **What broke for others.** - Syncthing, 2019: cleanout read the wrong timestamp and deleted a 2017 file on the day it was archived. The fix keeps the archive time in the version's name. - Nextcloud AIO users only discovered the 30-day pruning when they needed a restore. - restic and borg warn that mixing unrelated histories breaks retention. **Options rejected.** - (b) S3 metadata, or posix mtime/xattr: fails in S3 put and none modes, needs a HEAD per entry on every listing, and is lost on moves. - (c) A sidecar per chain: a metadata index with a read-modify-write race. - (d) Aging per origin: still needs a key change and is less precise. - (e) Document only: leaves the silent losses in place. **Recommendation: (a).** At merge time, write the replacement time into the version's key: `<12 digits>r<unix>[a|u]`, 98 bytes at most. - Works on posix and every S3 mode, needs no index or leases, and survives backups. - v0.6.347 and v0.6.348 already read these names, so a rollback is safe. - The simple rule from batch-2 research is not enough. In a simulation of 20,000 random merges it still aged 12,061 entries too early. - The corrected algorithm pins A's versions for free and pins B's versions only where needed, before A's arrive. It produced 0 early expiries in 120,000 checked entries, at about 1.5 renames per merge. **What users would see.** Every version of both files is kept for 90 days from when it was replaced. A few destination versions get new IDs, so a versions panel that was open during the move needs a refresh. **Side finding.** VERSIONING §7 says Nextcloud drops history on copy. Current Nextcloud copies it, so the docs need fixing. ## Decision needed 1. Adopt (a), replacement time in the key at merge? 2. Keep the 87-byte name guard? A version whose name would not fit stays unpinned and logs a WARN. 3. Accept a one-hour tolerance? Uploads that write a temp file and move it over the original then cost no extra rename, at the price of a version expiring up to one hour early.
Author
Owner

Decided by Nikola (2026-10-17):

  • When histories merge, the replacement time is written into the version's name, using the corrected algorithm from the research.
  • The 87-byte name guard stays. A version whose name would not fit stays unpinned and logs a WARN.
  • A one-hour tolerance applies.

Implementation: batch 2.

**Decided by Nikola (2026-10-17):** - When histories merge, the replacement time is written into the version's name, using the corrected algorithm from the research. - The 87-byte name guard stays. A version whose name would not fit stays unpinned and logs a WARN. - A one-hour tolerance applies. Implementation: batch 2.
Cordy closed this issue 2026-10-17 20:40:31 +00:00
Author
Owner

Shipped in v0.6.349 (PR #959). Both dogfoods run it.

When two histories merge, Cairn now writes each version's replacement time into its name, using the corrected algorithm (a pinned key: r<10-digit unix>, at most 98 bytes).

  • The destination's versions are pinned before the moved content is published, and pruning runs after the merge.
  • A pin never changes once written.
  • If the publish fails, the destination's current content gets a fresh record. A pinned duplicate takes no count slot, but only where identity is exact.
  • In the pin model, 0 contents expire early, including when a publish fails.

What remains is documented in VERSIONING §6/§7: the one-hour cases, saves that are still in flight, and #958 (overlapping saves published out of key order). Live functional checks on the dogfoods come after the next sign-in.

**Shipped in v0.6.349 (PR #959). Both dogfoods run it.** When two histories merge, Cairn now writes each version's replacement time into its name, using the corrected algorithm (a pinned key: `r<10-digit unix>`, at most 98 bytes). - The destination's versions are pinned before the moved content is published, and pruning runs after the merge. - A pin never changes once written. - If the publish fails, the destination's current content gets a fresh record. A pinned duplicate takes no count slot, but only where identity is exact. - In the pin model, 0 contents expire early, including when a publish fails. What remains is documented in VERSIONING §6/§7: the one-hour cases, saves that are still in flight, and #958 (overlapping saves published out of key order). Live functional checks on the dogfoods come after the next sign-in.
Author
Owner

Live status: not checked live. The fix only shows over time: whether merged versions expire at the right moment takes days to weeks, so it is covered by the rig tests and the property simulation (posix, posix-encrypt, s3, s3-encrypt, copy mode). On the dogfoods, the v0.6.349 live check confirmed that moves keep every version's content (PR #959). The remaining ordering case found later is #958.

**Live status:** not checked live. The fix only shows over time: whether merged versions expire at the right moment takes days to weeks, so it is covered by the rig tests and the property simulation (posix, posix-encrypt, s3, s3-encrypt, copy mode). On the dogfoods, the v0.6.349 live check confirmed that moves keep every version's content (PR #959). The remaining ordering case found later is #958.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#946
No description provided.