Upload a whole folder (picker and drag and drop, also on drop links) #1098

Closed
opened 2026-11-04 03:10:46 +00:00 by Cordy · 5 comments
Owner

Gap (v0.7 feature-gap research, 2026-11-04, decided by Nikola). The web UI uploads files only: there is no folder picker (webkitdirectory) and dropping a folder does not walk its entries (webkitGetAsEntry); neither appears in web/static/index.html at 0e0885db (VERIFIED).

Competitors. OpenCloud, Nextcloud, ownCloud 10 and Seafile upload folders with their structure from the web UI, including on public upload links (OpenCloud docs).

To research and decide when worked on. Creating the folder tree before the files; name conflicts at each level; progress for many files; partial failures and retry; empty folders; quota and per-file limits checked early; drop links and add-only devices (#900); virus scanning; TUS resumption across many files; accessibility of the picker and the drop zone.

Refs: scratch/b7e/backlog/gap-v07/research-v07-gaps.md A2, research-opencloud.md.

**Gap (v0.7 feature-gap research, 2026-11-04, decided by Nikola).** The web UI uploads files only: there is no folder picker (`webkitdirectory`) and dropping a folder does not walk its entries (`webkitGetAsEntry`); neither appears in `web/static/index.html` at 0e0885db (VERIFIED). **Competitors.** OpenCloud, Nextcloud, ownCloud 10 and Seafile upload folders with their structure from the web UI, including on public upload links ([OpenCloud docs](https://docs.opencloud.eu/docs/user/files-and-folders/upload-download-unzip)). **To research and decide when worked on.** Creating the folder tree before the files; name conflicts at each level; progress for many files; partial failures and retry; empty folders; quota and per-file limits checked early; drop links and add-only devices (#900); virus scanning; TUS resumption across many files; accessibility of the picker and the drop zone. Refs: `scratch/b7e/backlog/gap-v07/research-v07-gaps.md` A2, research-opencloud.md.
Author
Owner

Research done (scratch/b7e/backlog/research-1098.md). The server already has what a folder upload needs; this is mostly web app work.

  • Upload button split into Upload files and Upload folder (webkitdirectory); drops read every item with webkitGetAsEntry() before the first await and call readEntries until empty (Chromium returns 100 at a time). Drops recreate empty folders; the picker cannot.
  • Plan before anything is sent (#975): skip a fixed list of system files (.DS_Store, ._*, Thumbs.db, desktop.ini) and count them; refuse names Cairn will not accept up front with the reason; a folder onto an existing folder gets Add to it (uploads only), each clashing file inside is asked about; room and scan checks on the whole tree; confirmation above 1,000 files or 10 GB.
  • Folders first, then files, through the existing one-at-a-time sender and TUS; a batch progress line, one result line, a Not uploaded list with Try again and Copy list.
  • Server: conflict=keepBoth on POST /api/v1/dirs (the server picks "Photos 2"); drop links accept a relative path confined to the link's folder.

Found on the way: the drop handler skips 0-byte files without a known type (index.html:19187, from reading the code); a visitor's upload through a drop link replaces an existing file of the same name at the link's root (public.go:453-459, from reading the code). Proposed: drop links never replace, taken names get the next number.

Decisions asked of Nikola next.

**Research done** (`scratch/b7e/backlog/research-1098.md`). The server already has what a folder upload needs; this is mostly web app work. - Upload button split into **Upload files** and **Upload folder** (`webkitdirectory`); drops read every item with `webkitGetAsEntry()` before the first `await` and call `readEntries` until empty (Chromium returns 100 at a time). Drops recreate empty folders; the picker cannot. - Plan before anything is sent (#975): skip a fixed list of system files (`.DS_Store`, `._*`, `Thumbs.db`, `desktop.ini`) and count them; refuse names Cairn will not accept up front with the reason; a folder onto an existing folder gets **Add to it** (uploads only), each clashing file inside is asked about; room and scan checks on the whole tree; confirmation above 1,000 files or 10 GB. - Folders first, then files, through the existing one-at-a-time sender and TUS; a batch progress line, one result line, a **Not uploaded** list with Try again and Copy list. - Server: `conflict=keepBoth` on `POST /api/v1/dirs` (the server picks "Photos 2"); drop links accept a relative path confined to the link's folder. Found on the way: the drop handler skips 0-byte files without a known type (`index.html:19187`, from reading the code); a visitor's upload through a drop link replaces an existing file of the same name at the link's root (`public.go:453-459`, from reading the code). Proposed: drop links never replace, taken names get the next number. Decisions asked of Nikola next.
Author
Owner

Decisions (Nikola, 2026-11-04): all recommendations approved. "Add to it" for uploads only (moves and copies never merge), each clashing file asked about; conflict=keepBoth on folder create; system files skipped and counted, no switch; drop links never replace (taken names get the next number), fixed in the same release; folders on every drop link; confirmation above 1,000 files or 10 GB; one file at a time in v1. Next: whole-page mockup.

**Decisions (Nikola, 2026-11-04): all recommendations approved.** "Add to it" for uploads only (moves and copies never merge), each clashing file asked about; `conflict=keepBoth` on folder create; system files skipped and counted, no switch; drop links never replace (taken names get the next number), fixed in the same release; folders on every drop link; confirmation above 1,000 files or 10 GB; one file at a time in v1. Next: whole-page mockup.
Author
Owner

Mockup approved (Nikola, 2026-11-04): 1098-folder-upload-mockup.html, frames A to K, its strings, the five open points as recommended (room checked twice; Stop link; no extra resume UI, handbook advice; skip-same-size-and-date unticked with count; inline big-folder confirmation on drop links) and the proposed server details (multipart mkdir on the link page, codes name-invalid and folder-missing, no new audit verbs). Next: spec and plan (.superpowers/sdd/2026-11-04-fu1098).

**Mockup approved (Nikola, 2026-11-04):** `1098-folder-upload-mockup.html`, frames A to K, its strings, the five open points as recommended (room checked twice; Stop link; no extra resume UI, handbook advice; skip-same-size-and-date unticked with count; inline big-folder confirmation on drop links) and the proposed server details (multipart `mkdir` on the link page, codes `name-invalid` and `folder-missing`, no new audit verbs). Next: spec and plan (`.superpowers/sdd/2026-11-04-fu1098`).
Author
Owner

Spec open points decided (Nikola, 2026-11-04), after research (research-1098-unsettled.md): all verdicts approved. Notably: the drop-link big-folder confirm adds "Keep this page open until the upload is done." and a several-items variant; on the public rate limit the page waits the seconds left and shows "Waiting {s} seconds…"; FR/IT whole-sentence singular keys; "Ce nom est réservé à Cairn." / "Questo nome è riservato a Cairn."; the skip rule is same size and not older, with the date comparison fixed. The S3 visitor folder race and adding into existing subfolders are documented, not blockers.

**Spec open points decided (Nikola, 2026-11-04), after research (`research-1098-unsettled.md`):** all verdicts approved. Notably: the drop-link big-folder confirm adds "Keep this page open until the upload is done." and a several-items variant; on the public rate limit the page waits the seconds left and shows "Waiting {s} seconds…"; FR/IT whole-sentence singular keys; "Ce nom est réservé à Cairn." / "Questo nome è riservato a Cairn."; the skip rule is same size and not older, with the date comparison fixed. The S3 visitor folder race and adding into existing subfolders are documented, not blockers.
Cordy closed this issue 2026-11-04 21:48:12 +00:00
Author
Owner

Shipped in v0.6.398 (PR #1118), live on both dogfoods (posix and s3: cairnd listening … version=v0.6.398, healthy, no ERROR at start; POST /api/v1/dirs?conflict=keepBoth without a session answers 401).

What's in it: Upload files / Upload folder split button, folder drops, a plan before anything is sent (refused names, the large-folder question incl. "Upload 1 file", one question per top-level item with Add to it), progress with Stop, a Not uploaded list with Try again; upload links with Choose a folder and folder drops, Keep both for files and folders, never replacing, paths confined to the link's folder, .trash refused; REST conflict=keepBoth on POST /api/v1/dirs, name-invalid on REST and TUS. Fix: 0-byte untyped files dropped on their own upload again. Behaviour change for scripts that PUT to upload links: CHANGELOG Security.

Strings approved 2026-11-04 (upWhyStopped, IT «Aggiungi alla cartella», upBigTitleOne, the link page's "Upload 1 file" and empty-folder line). Reviews: W1+W2, W3, final, three fix rounds. Residuals (#742): Keep both on s3 is serialized per parent within one process only, and link vs REST use separate locks; Stop does not cancel a running small POST; link-page drops keep no file dates. Follow-ups are backlog lines.

Closing. Dogfood checks go on #1005.

Shipped in **v0.6.398** (PR #1118), live on both dogfoods (posix and s3: `cairnd listening … version=v0.6.398`, healthy, no ERROR at start; `POST /api/v1/dirs?conflict=keepBoth` without a session answers 401). What's in it: Upload files / Upload folder split button, folder drops, a plan before anything is sent (refused names, the large-folder question incl. "Upload 1 file", one question per top-level item with **Add to it**), progress with Stop, a Not uploaded list with Try again; upload links with Choose a folder and folder drops, Keep both for files and folders, never replacing, paths confined to the link's folder, `.trash` refused; REST `conflict=keepBoth` on `POST /api/v1/dirs`, `name-invalid` on REST and TUS. Fix: 0-byte untyped files dropped on their own upload again. Behaviour change for scripts that PUT to upload links: CHANGELOG Security. Strings approved 2026-11-04 (upWhyStopped, IT «Aggiungi alla cartella», `upBigTitleOne`, the link page's "Upload 1 file" and empty-folder line). Reviews: W1+W2, W3, final, three fix rounds. Residuals (#742): Keep both on s3 is serialized per parent within one process only, and link vs REST use separate locks; Stop does not cancel a running small POST; link-page drops keep no file dates. Follow-ups are backlog lines. Closing. Dogfood checks go on #1005.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#1098
No description provided.