Repository navigation
Conversation
Local Windows 11 x86_64:
Not run at 9de0658: WSL (Linux) and macOS. At 6946bc9, before the merge, WSL clippy (all features and no default features) and The merge needed two fits to The |
[High impact] Adds file transfer over SSH with new remote file APIs. Fix the download-completion race before merging so saved edits cannot silently go unnoticed. FindingsSummaryAdds SSH file downloads, per-window cached copies, save-upload questions, conflict choices, and the read-only
leftos acknowledges these deferred limits: unsent edits are deleted at quit without warning; host-named Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Click SSH file path] --> B[Download into window copy]
B --> C[Open copy]
C --> D[Watch saved changes]
D --> E[Ask before upload]
E --> F[Check host file hash]
F -->|Matches| G[Upload saved bytes]
F -->|Changed| H[Show conflict choices]
H --> I[Keep both]
H --> J[Overwrite anyway]
H --> K[Re-fetch]
Reviews (2) · Last reviewed commit: "feat: list fetched SSH file copies with ..." · Reviewed by Greptile |
b40727a to
2d05c91
Compare
2d05c91 to
7d9b952
Compare
| let synced_meta = std::fs::metadata(&local) | ||
| .and_then(|metadata| watch::Meta::of(&metadata)) |
There was a problem hiding this comment.
fetch_copy reads synced_meta from the destination after replacing it. If an editor saves between the rename and this read, the copy is recorded as Synced with the edit's metadata but the downloaded bytes' hash. The watcher then sees unchanged metadata and never asks to upload that save.
Capture the downloaded file's metadata from the part file before renaming it, so a later editor save remains detectable.
There was a problem hiding this comment.
🤖 Posted by Claude Code on behalf of @leftos.
Fixed in 235791d: fetch_copy now reads the metadata from the whole part file before the rename. watch::Meta holds only mtime and length, and a rename keeps both, so the recorded metadata is the download's. A save made as the copy lands then differs from it and leads to the upload question. remote_files/tests/fetch_meta.rs saves to the copy right after the rename and failed before the change.
| let orphans: Vec<Key> = self | ||
| .copies | ||
| .iter() | ||
| .filter(|(_, copy)| matches!(copy.state, SyncState::Asking { .. })) |
There was a problem hiding this comment.
Pressing Escape on the conflict dialog removes its only queued question but leaves the copy in Conflict. requeue_unanswered restores only upload questions, and clicking the path just reopens the edited copy. Users cannot choose Keep both, Overwrite anyway, or Re-fetch again without another save.
Keep a way to reopen dismissed conflicts, such as reopening their question when the path is clicked.
There was a problem hiding this comment.
🤖 Posted by Claude Code on behalf of @leftos.
Fixed in 235791d: a click on the path of a copy in conflict still opens the copy, and now queues its conflict question again when it is neither waiting nor showing, so Keep both, Overwrite anyway, and Re-fetch come back after Escape. Two clicks queue it once. remote_files/tests/dismissed_conflict.rs covers both, and that a click on a copy not in conflict queues nothing.
d5c983d to
ca502c5
Compare
Add herdr_client::remote_file, which copies a file off an SSH host and writes it back over the endpoint's noninteractive SSH trust, for the upcoming remote file links in the GUI. POSIX hosts run /bin/sh scripts; Windows OpenSSH hosts run Windows PowerShell 5.1+ scripts through -EncodedCommand. The path travels quoted in the script; file bytes travel only on stdout and stdin. Both shells share one exit-code contract for missing, not-a-file, too-large, symlink, no-hasher, and conflict refusals. A push writes a private temporary file beside the target and renames it over the target only while the host's file still matches the expected SHA-256 (or is absent, or anything). An existing POSIX file keeps its permission bits; a new one gets the login umask's default. Tests run the POSIX scripts through a local /bin/sh and, opt-in, the PowerShell scripts through a local powershell.exe and pwsh.
A link-modifier click on a path printed in a pane on an SSH device did nothing, since this machine cannot open that host's paths. Such paths are now links by their shape: a click fetches the file over the device's SSH trust into a per-launch cache and opens the copy in the system's application, or the folder holding it when it is not a known document, under the same rule as local paths. Windows OpenSSH hosts are recognised by a drive or UNC path, or a Windows working directory for a relative path, and use PowerShell; other hosts use /bin/sh. Relative paths join the pane's cwd with the host's separator, and `~` is left for the host to expand. Hover never contacts the host. Plugin link handlers keep the first claim. WSL and cloud panes are unchanged, as are SSH panes on Windows clients, which have no SSH children. The cache is removed at quit, and folders left by earlier launches are swept on first use.
A copy fetched by clicking a path in an SSH pane is now watched for saves. The first save asks whether to upload it: Upload, Upload future saves without asking, or Not now. Later saves follow that answer, and an automatic upload reports itself in a flash. Every upload sends the hash the host's file had when it was last synced. When the file changed on the host since then, nothing is written and a dialog offers Keep both (uploads beside it as name.conflict.ext), Overwrite anyway, or Re-fetch. Copies over 64 MB are not uploaded. The client's push now carries the exact byte count, and the host refuses to replace its file with a temp file that holds fewer bytes, so a dropped connection or cancelled upload leaves it unchanged. A click that fetches a copy again keeps a save made while the fetch ran. The host, shell, and path of a transfer travel as one RemoteFile. Watching runs off the UI thread at most once a second; a change counts once two runs agree on it, which lets an editor's atomic save settle. Prompts wait for any open dialog instead of replacing it. Saving a copy back to its synced bytes drops its question and closes its dialog, and a failed Re-fetch offers the conflict again. An upload never makes the file readable by anyone who could not read it before. A POSIX host gives the temp file the target's group and mode before the rename, and writes in place instead when the group cannot be kept with the same reach or an ACL shows (`ls` `+` or `@`, `getfacl`). Windows PowerShell 5.1 and 7 create the temp file with the target's DACL. Keep both takes the original file's access. When access cannot be kept and nothing can be written safely, the host exits with its own code and the user is told so. A macOS host's inheritable folder ACL can still read the temp file during the transfer. Each window keeps its copies in its own cache folder, so a fetch in one window never replaces another's unsent edits. A click while its file is still being fetched starts no second fetch, and each fetch writes its own part file.
`herdr-gpui remote-files [list]` reports the copies of SSH hosts' files that link clicks fetched into the window, each with its host, host path, local path, sync state, and whether its saves upload without asking. It is read-only: uploads stay behind the app's dialog. The reply fits the socket's message budget and counts what it omits, as `notes list` does. Document remote file links in the Herdr UI README, the transfer API in the client README, the command in docs/cli.md, and the behaviour beside open_files_in in the example config.
The overview said SSH needs "an installed POSIX Herdr", but a macOS client connects to a Windows OpenSSH host running Herdr 0.9.4, and the Files on SSH Hosts section already describes Windows hosts.
ca502c5 to
5945c64
Compare
# Conflicts: # crates/herdr-gpui/src/menu/chrome.rs # crates/herdr-gpui/src/menu/chrome/keys.rs
Two fixes from review of the SSH file copies: - A repeat fetch checked the copy for saves, then renamed the downloaded file over it, so a save landing between the two was overwritten and the copy marked synced. The copy now moves aside first, the download is hard-linked into its place, which fails rather than replace a save that took the place meanwhile, and the copy moved aside comes back unless its metadata proves it is the one last synced. A filesystem without hard links falls back to a rename. The watch run no longer probes a copy a fetch is writing, and a conflict's refetch now holds its copy in the in-flight set as a click's fetch does, so only one writer handles a copy at a time. - A copy's file name was split at both separators for every host, so /srv/foo\bar.txt on a POSIX host was cached and shown as bar.txt. It is now split at the host shell's own separators, and on Windows the backslash becomes an underscore as the local naming rules intend.
The POSIX push script checked the target for a symbolic link, then reopened it by name: for its temporary file, for chgrp and chmod, and for the in-place write it falls back to when a file's ACL cannot be kept. Another user who can rename entries in the folder could swap in a link between the check and the write, so the upload landed in a different file the SSH account can write. The script now: - enters the target's folder once with cd -P and names the target relative to it, so no later step re-resolves the folder's path; - builds the copy in a private mktemp -d folder inside it, checked to be the SSH user's drwx------ folder within the anchored one, with every file in it created by mktemp, so nothing that already exists is opened or replaced, and moves it over the target with mv -T (GNU) or mv -h (BSD) where mv takes one; - writes in place only in a folder no one else can replace files in (owned by the SSH user or root, no ACL, and not writable by group or others unless sticky with the target the user's or root's), and otherwise refuses with a new exit code, shown as "Other users can replace files in that folder on the host, so it was not written in place; nothing was changed."; - never writes in place for Keep both, and re-checks that the new name is still free just before the rename. A push now also needs id, ls and rmdir on the host.
A POSIX push renames a copy owned by the SSH user over the target, so a target another account owned becomes the SSH user's. The copy kept the target's mode, set-user-ID bit included: as root, pushing over another account's 4755 file produced a root-owned setuid executable that runs as root for everyone. Keep the set-user-ID bit only when the copy's owner matches the file whose access it keeps; an unreadable owner on either side drops it. Drop the set-group-ID bit when chgrp fails and the copy keeps its own group. Sticky stays: on a regular file it names no owner or group.
# Conflicts: # crates/herdr-gpui/src/cli/control/hosts.rs # crates/herdr-gpui/src/control/client/hosts.rs # crates/herdr-gpui/src/control/protocol/hosts.rs # crates/herdr-gpui/src/control/protocol/hosts/tests.rs # crates/herdr-gpui/src/control/server/hosts.rs # crates/herdr-gpui/src/control/server/hosts/tests.rs # crates/herdr-gpui/src/menu/chrome.rs # crates/herdr-gpui/src/menu/chrome/keys.rs
Summary
A link-modifier click on a file path printed in a pane on an SSH device did nothing:
local_link_at(window/file_links.rs) dropped every path on a remote pane, since this machine cannot open that host's files. On macOS and Linux clients such a path is now a link, and the click opens the host's file here.remote_files/cache.rs), one subfolder per window, and opens the copy under the same rule as a local path: a document opens in its default application, anything else opens the folder holding it. Files up to 64 MB; a folder, a larger file, or a missing one shows why nothing opened. Each window keeps its own copies, so a fetch in one window never replaces another's unsent edits, and a second click while a fetch runs starts nothing.-EncodedCommand; anything else picks/bin/sh. Relative paths join the pane's cwd with the host's separator;~expands on the host. This builds on fix: make Windows drive paths in pane text clickable #610's drive-path reading.notes.conflict.mdbeside it), Overwrite anyway, or Re-fetch. A failed Re-fetch offers the conflict again, and saving the copy back to its synced bytes drops the question and closes its dialog. A conflict dialog closed without an answer comes back when the path is clicked again, which still opens the copy. A fetch records the downloaded file's metadata before putting it in place, so a save made as the copy lands is still noticed. A repeat fetch that keeps edits moves the copy aside, hard-links the download into its place (which fails rather than replace a save that took the place), and puts the copy back unless its metadata proves it unsaved; the watch skips a copy a fetch is writing, and a conflict's Re-fetch holds the copy as a click's fetch does.herdr-client/src/remote_file*). A push writes a temp file beside the target and renames it over the target only when it holds exactly the expected byte count and the host's file still matches; symlinks and non-regular files are never replaced. File bytes travel only on stdin/stdout, and the untrusted path only as a quoted literal in the script.cd -P), builds the copy in a privatemktemp -dfolder inside it (checked to be the SSH user'sdrwx------folder there, every file in it made bymktemp), gives it the target's group and mode, and moves it over the target withmv -T/mv -hwheremvtakes one. It writes in place instead when the group cannot be kept with the same reach or an ACL shows (ls+, an ACL listed by macOSls -lde, orgetfacl -s), but only in a folder no one else can replace files in (owned by the SSH user or root, no ACL, and not writable by group or others unless sticky with the target the user's or root's); elsewhere it refuses: "Other users can replace files in that folder on the host, so it was not written in place; nothing was changed." An unwritable target is refused with its own exit code, and the user sees "Could not keep the file's access; nothing was changed." Keep both never writes in place and re-checks the new name just before the rename. The host needsid,lsandrmdirbesides the tools it already used. Windows PowerShell 5.1 and 7 create the temp file with the target's DACL. Keep both gives the conflict copy the original file's access (RemoteFileAccess::Like).herdr-gpui remote-files [list]reports the copies a window holds and their sync state, per the Command Line rule. It is read-only.mainruns SSH children on Windows (feat(windows): add and connect to saved SSH hosts on Windows #624, 1bf759b), the client builds the samesshcommand for POSIX and Windows OpenSSH hosts on every OS, and SSH-pane path links are no longer limited to unix clients. A host file whose name Windows cannot store (<>:"/\|?*, a control character, a trailing dot or space, or a device name such asCONorcom1.txt) gets a safe local copy name (a:b.txtbecomesa_b.txt); dialogs and uploads keep the host's name and path. Without the rule,a:b.txtjoined onto the cache folder is a drive-relative path outside it. On Windows the cache folder inherits the ACL of the user's state folder, as the app's other private files do.Validation
Rebased onto
mainat 350e3e0. Run at 5945c64's tree on Windows 11 x86_64, and in WSL Ubuntu (Linux x86_64) on the same tree (earlier rounds' results below are marked with their commit):check-commits(upstream/main..HEAD): okjust ci(local):format-check,check-file-size,check-sorted, bothlintclippy passes, andcargo test --locked --workspacedefault and all features green (herdr-gpui 3793 and 3836, herdr-client 229, 0 failed). The un-gatedwindow/file_links/testsremote and remote-save tests now run on Windows.cargo test --workspace: passed, 0 failed (4241 passed). herdr-clientremote_file --include-ignored: 41 passed.remote_filetests on Windows (each script underpowershell.exe5.1 andpwsh7): 23 passed.remote_files/tests/windows_names.rsfailed before the Windows name rule (a<b.txtkept; the copy path escaped asb.txt).just lint-linuxandjust lint-windows.just test-mockup-wait: fails on Windows only, as in fix: make Windows drive paths in pane text clickable #610 (test_wait.pyhandsbasha Windows path). This PR touches nothing under.claude/.cargo clippy --locked --workspace --all-targets --all-features -- -D warningsand--no-default-features: ok (covers thecfg(unix)code Windows does not build)cargo test --locked --workspace, default features: herdr-gpui 4169 passed, 1 failed:orchestrator::tests::service::the_worker_shows_runs_another_application_writestimed out waiting for a snapshot under load. This PR touches no orchestrator code, and the test passes run alone. The all-features WSL run was not repeated after the rebase.remote_files/tests/fetch_meta.rs(a save as the fetched copy lands is still asked about) andremote_files/tests/dismissed_conflict.rs(a click on a conflicted copy's path asks its closed question again, once) failed before their fixes.leftos/patches(which carries this PR at 2d05c91) on Apple silicon, macOS 27.0.1:cargo build --locked --release -p herdr-gpuiandjust bundlesucceeded.TMPat a junction (the/var -> /private/varcase):a_script_opens_the_folder_holding_its_copyfailed with the CI assertion before the fix and passes after, with and without the junction.remote_files/tests/fetches.rs), per-window copies, theAsking/Conflict+Revertedand failed Re-fetch rows in the step table, the stale-dialog close, and Keep both sending the original's access.remote_filein WSL with--include-ignored: 40 passed, including the ACL cases (setfaclon the target, a folder default ACL, the group rules for 0644/0640/0604/0664,Likesources that are links or folders, and the access-refusal exit code).powershell.exe5.1 andpwsh7.6, before the rebase (at 2d05c91):powershell_scripts_fetch_and_push_files,powershell_push_temporary_file_takes_the_targets_aclandpowershell_push_kept_copy_takes_its_sources_aclpassed.remote_file44 passed andposix_exec -- --ignored2 passed (tool shims swap the target, the temp file and the work folder for links; a file planted before Keep both's rename; a folder ACL),remote_files/tests/fetch_swap.rs(saves in place, by temp-and-rename and through a handle at each step of the swap, the in-flight guard), and the backslash name tests. Thenjust cion Windows (all buttest-mockup-wait) and WSL clippy ×2 andcargo test --workspacedefault and all features at 6a31f50, all passed. On macOS 27.0.1 (Apple silicon) at 6a31f50, under its/bin/shand BSD tools: herdr-clientremote_file44 passed and herdr-gpuiremote_files58 passed.Screenshots
None attached yet: the first manual checks below ran on a MacBook, but no screenshots of the two dialogs or the underline on a remote pane were captured.
Known limitations
file://<host>/pathhyperlink naming the SSH host itself, asls --hyperlinkprints, still answers "on another host" (terminal/links.rsexplicit_target). Printed paths andfile:///pathlinks fetch.File.Replacefailure partway (error 1177) can leave the Windows target in an unclear state.mvtakes neither-Tnor-h(older BusyBox), a link to a folder planted at the target in the moment before the rename makesmvmove the copy into that folder; the target is unchanged.Manual QA
echo C:\dev\x\notes.mdin a PowerShell pane, hold Cmd over it (solid underline), click: the copy opens in its default app. (MacBook Air, macOS 27.0.1, against a Windows 11 host running Herdr 0.9.4, with anotes.md:5under the user's profile;leftos/patchesbuild.)type C:\dev\x\notes.mdon the host. (Same setup: the edit arrived on the host, line endings kept.)src\main.rsunder the right cwd, a folder, and a file over 64 MB.~/notes.mdand a relative path open; an executable script opens its folder, and a push keeps its mode.0640file of a group you belong to (not your primary group), and a file with asetfaclentry, both keep their group and ACL after an upload. Keep both on a0600.envgives a0600conflict copy.herdr-gpui remote-fileslists the copies; quit, and the cache folder under~/.local/state/herdr/gpui/remote-files/is gone.~/notes.mdopens the copy; save and Upload; the file on the host changes. A host file nameda:b.txtopens asa_b.txt, and its upload goes toa:b.txt.C:\dev\x\notes.mdopens the copy, and an upload lands.chmod +aACL entry; the first lands, the second is written in place, and no.notes.md.XXXXXXfolder is left beside it.chmod 1777folder and in achmod 777folder, upload to a file with asetfaclentry; the sticky folder takes the in-place write, the other refuses with "Other users can replace files in that folder on the host…" and the file is unchanged.