Versions: files created by a folder copy are unrecorded until their first save #950

Closed
opened 2026-10-17 02:01:24 +00:00 by Cordy · 2 comments
Owner

From the fix-922 spec §14 (2026-10-16), rule R2.

What happens: since v0.6.347 every save and every file copy records its content before publishing it. A folder copy still copies its files with the driver's own copy and records none of them. Each copied file gets its record at its first save.

On one Cairn instance this loses nothing, because the first save records the content it replaces. If several instances ever share one storage, a save on another instance racing the folder copy could replace a just-copied file before anything recorded it. This is listed in VERSIONING.md section 7a as an accepted window.

To do (to decide):

  • Either record each copied file during the folder copy. This costs one record per file: a hard link on posix, a CopyObject on s3.
  • Or keep the window and document it, as now.

Add a twin test either way.

From the fix-922 spec §14 (2026-10-16), rule R2. **What happens:** since v0.6.347 every save and every file copy records its content before publishing it. A folder copy still copies its files with the driver's own copy and records none of them. Each copied file gets its record at its first save. On one Cairn instance this loses nothing, because the first save records the content it replaces. If several instances ever share one storage, a save on another instance racing the folder copy could replace a just-copied file before anything recorded it. This is listed in VERSIONING.md section 7a as an accepted window. **To do (to decide):** - Either record each copied file during the folder copy. This costs one record per file: a hard link on posix, a CopyObject on s3. - Or keep the window and document it, as now. Add a twin test either way.
Author
Owner

Built in the #742 Step 0 batch (st742-t17), decision (Nikola, 2026-11-06): a folder copy records each copied file before it is published (twin race closed). Two consequences came up in the build and were decided:

  • Keep each file's version history on a folder copy (the old code copied the histories; the first build of t17 dropped them). The fix round restores that: histories travel with the copy, and each copied file is recorded on top.
  • S3 cost: a 100-file folder copy now costs about 3,100-3,650 storage requests instead of 129-843 (each file goes through the full single-file copy checks). Optimise before v1.0: follow-up filed to batch the per-file checks per folder. POSIX adds one hard link per file.

Also accepted (2026-11-06) in Step 0: the group pass takes over at once after a crash restart with a different run; an unreadable busy-lease record refuses with 503 busy-unavailable; a refused manual lock race answers 409 lock-held.

**Built in the #742 Step 0 batch (st742-t17), decision (Nikola, 2026-11-06):** a folder copy records each copied file before it is published (twin race closed). Two consequences came up in the build and were decided: - **Keep each file's version history on a folder copy** (the old code copied the histories; the first build of t17 dropped them). The fix round restores that: histories travel with the copy, and each copied file is recorded on top. - **S3 cost:** a 100-file folder copy now costs about 3,100-3,650 storage requests instead of 129-843 (each file goes through the full single-file copy checks). Optimise before v1.0: follow-up filed to batch the per-file checks per folder. POSIX adds one hard link per file. Also accepted (2026-11-06) in Step 0: the group pass takes over at once after a crash restart with a different run; an unreadable busy-lease record refuses with 503 `busy-unavailable`; a refused manual lock race answers 409 `lock-held`.
Author
Owner

Shipped in v0.6.401 (PR #1138), live on both dogfoods. A folder copy records each copied file before it is published (the twin race is closed) and, as you decided, keeps each file's version history: the source chain is copied after the file lands, re-encrypted for a copy into another key scope; an entry that cannot be copied is skipped with one WARN per folder copy (versions: version not copied with the file). Twin and history tests on posix and s3 in every mode.

Cost: one extra CopyObject per older version on S3 (100 files written once: 3,650 to 4,150 requests on the test double). Optimising that is #1135 (v0.9). Timing a real folder copy on the S3 dogfood is on the #1005 list.

Shipped in **v0.6.401** (PR #1138), live on both dogfoods. A folder copy records each copied file before it is published (the twin race is closed) and, as you decided, keeps each file's version history: the source chain is copied after the file lands, re-encrypted for a copy into another key scope; an entry that cannot be copied is skipped with one WARN per folder copy (`versions: version not copied with the file`). Twin and history tests on posix and s3 in every mode. Cost: one extra CopyObject per older version on S3 (100 files written once: 3,650 to 4,150 requests on the test double). Optimising that is #1135 (v0.9). Timing a real folder copy on the S3 dogfood is on the #1005 list.
Cordy closed this issue 2026-11-06 15:22:40 +00:00
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#950
No description provided.