Define sshx agent execution contract and close E2E quality gaps

.github/workflows/ci.yml

@@ -25,7 +25,7 @@ jobs:
           go-version: ${{ matrix.go }}
 
       - name: Cache Go modules
-        uses: actions/cache@v3
+        uses: actions/cache@v4
         with:
           path: |
             ~/.cache/go-build
@@ -38,12 +38,12 @@ jobs:
         run: go mod download
 
       - name: Run tests
-        run: go test -v -race -coverprofile=coverage.out -covermode=atomic ./...
+        run: go test -v -short -race -coverprofile=coverage.out -covermode=atomic ./...
         shell: bash
 
       - name: Upload coverage to Codecov
         if: matrix.os == 'ubuntu-latest'
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v5
         with:
           files: ./coverage.out
           flags: unittests
@@ -52,6 +52,75 @@ jobs:
       - name: Build
         run: go build -v ./cmd/sshx
 
+  e2e:
+    name: E2E (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: "1.24"
+
+      - name: Download dependencies
+        run: go mod download
+
+      - name: Prepare isolated macOS keychain
+        if: runner.os == 'macOS'
+        shell: bash
+        run: |
+          keychain_path="$RUNNER_TEMP/sshx-e2e.keychain-db"
+          keychain_password="sshx-e2e-ci"
+          original_keychain="$(security default-keychain -d user | sed 's/^[[:space:]]*\"//; s/\"[[:space:]]*$//')"
+          keychain_list_file="$RUNNER_TEMP/sshx-e2e-original-keychains.txt"
+          security list-keychains -d user | sed 's/^[[:space:]]*\"//; s/\"[[:space:]]*$//' > "$keychain_list_file"
+          original_keychains=()
+          while IFS= read -r item; do
+            if [ -n "$item" ]; then original_keychains+=("$item"); fi
+          done < "$keychain_list_file"
+          security create-keychain -p "$keychain_password" "$keychain_path"
+          security set-keychain-settings -lut 3600 "$keychain_path"
+          security unlock-keychain -p "$keychain_password" "$keychain_path"
+          security list-keychains -d user -s "$keychain_path" "${original_keychains[@]}"
+          security default-keychain -d user -s "$keychain_path"
+          {
+            echo "SSHX_E2E_KEYCHAIN=$keychain_path"
+            echo "SSHX_E2E_ORIGINAL_KEYCHAIN=$original_keychain"
+            echo "SSHX_E2E_ORIGINAL_KEYCHAIN_LIST=$keychain_list_file"
+          } >> "$GITHUB_ENV"
+
+      - name: Run compiled-binary E2E tests
+        env:
+          SSHX_E2E_REAL_KEYRING: ${{ runner.os == 'macOS' && '1' || '0' }}
+        run: go test -v ./tests/e2e
+
+      - name: Remove isolated macOS keychain
+        if: always() && runner.os == 'macOS'
+        shell: bash
+        run: |
+          if [ -n "${SSHX_E2E_ORIGINAL_KEYCHAIN:-}" ]; then
+            security default-keychain -d user -s "$SSHX_E2E_ORIGINAL_KEYCHAIN"
+          fi
+          if [ -f "${SSHX_E2E_ORIGINAL_KEYCHAIN_LIST:-}" ]; then
+            original_keychains=()
+            while IFS= read -r item; do
+              if [ -n "$item" ]; then original_keychains+=("$item"); fi
+            done < "$SSHX_E2E_ORIGINAL_KEYCHAIN_LIST"
+            if [ ${#original_keychains[@]} -gt 0 ]; then
+              security list-keychains -d user -s "${original_keychains[@]}"
+            fi
+            rm -f "$SSHX_E2E_ORIGINAL_KEYCHAIN_LIST"
+          fi
+          if [ -n "${SSHX_E2E_KEYCHAIN:-}" ]; then
+            security delete-keychain "$SSHX_E2E_KEYCHAIN" || true
+          fi
+
   lint:
     name: Lint
     runs-on: ubuntu-latest

AGENT.md

@@ -10,36 +10,45 @@ Module: `github.com/talkincode/sshx` · Language: Go 1.24 · License: MIT
 
 ## 1. Mission
 
-`sshx` is a barrier-free, cross-platform **SSH/SFTP command-line client** with a
-built-in **OS-keyring password manager** and **named host configuration**. It
-exists to make ad-hoc operations across many remote servers fast and safe:
+`sshx` is an **agent-native remote host execution tool over SSH**. SSH/SFTP is
+the trusted transport; execution is the product. It turns an agent's intent
+into a one-shot remote operation whose target and side effects are explicit,
+whose result is machine-decidable, and whose security context is auditable:
 
-> One command, multiple servers, zero password hassle.
+> SSH is the channel. X is execution.
 
 The core value proposition:
 
-- Run a command (or transfer a file) on a remote host in a single invocation.
-- Never type or store passwords in plaintext — they live in the OS keyring and
-  sudo passwords are auto-filled.
-- Address hosts by a short name instead of full connection details.
-- Be secure by default (strict host-key verification, command safety guardrails).
+- Give agents one stable process contract for target resolution, preview,
+  execution, structured results, failure classification, and audit evidence.
+- Run a command or file action on an existing SSH host in one invocation, with
+  no resident remote agent and no long-running control plane.
+- Reduce decision and retry cost through named hosts, JSON, exit codes,
+  `error_kind`, timeout, and dry-run plans.
+- Protect credentials and trust boundaries through the OS keyring, strict
+  host-key verification, sudo over stdin, explicit bypasses, and audit redaction.
+
+Human operators use the same CLI, preview, safety, and audit semantics to
+supervise agents and troubleshoot operations.
 
 ## 2. Goals
 
-1. **Single self-contained binary** — no runtime dependencies, installable via
+1. **Stable agent execution contract** — predictable stdout/stderr, exit codes,
+   JSON results, failure kinds, previews, and audit semantics.
+2. **Single self-contained binary** — no runtime dependencies, installable via
    `go install`, an install script, or a downloaded release artifact.
-2. **Cross-platform parity** — Linux, macOS, and Windows are all first-class.
-3. **Secure by default** — strict `known_hosts` verification, keyring-backed
+3. **Cross-platform parity** — Linux, macOS, and Windows are all first-class.
+4. **Secure by default** — strict `known_hosts` verification, keyring-backed
    secrets, sudo password delivered over stdin (never interpolated), and command
    safety checks that block obviously destructive operations.
-4. **Low cognitive load** — sensible defaults, named hosts, key-first auth with
+5. **Low agent decision cost** — sensible defaults, named hosts, key-first auth with
    password fallback only when an SSH login password is already provided, and
-   helpful error messages.
-5. **Multi-server ergonomics** — per-host SSH keys and per-host/per-server
+   classified failures rather than prose-only errors.
+6. **Multi-server ergonomics** — per-host SSH keys and per-host/per-server
    password keys so one tool covers a whole fleet.
-6. **Execution preview** — `--dry-run` explains the local execution plan without
+7. **Execution preview** — `--dry-run` explains the local execution plan without
    connecting, executing, reading keyring secrets, or mutating state.
-7. **Auditability** — non-dry-run invocations write structured JSONL audit
+8. **Auditability** — non-dry-run invocations write structured JSONL audit
    events under `~/.sshx/audit` by default, with secrets and stdout/stderr
    excluded.
 
@@ -55,6 +64,11 @@ the project's mission:
   CLI-only. Do not reintroduce an `mcp-stdio` mode or MCP tools.
 - ❌ **Daemons / long-running services / connection pools** — every command opens
   a connection, does its work, and exits. There is no background process.
+- ❌ **Resident remote agent / control plane** — do not require a service to be
+  installed on managed hosts and do not turn sshx into a fleet control plane.
+- ❌ **Desired-state configuration / workflow orchestration** — bounded fan-out
+  execution is in scope; playbooks, schedulers, reconciliation, and long-lived
+  workflow state are not.
 - ❌ **GUI / TUI** — interaction is through flags and stdout/stderr only.
 - ❌ **Full OpenSSH replacement** — no interactive login shell multiplexing,
   port forwarding / tunneling, SOCKS proxy, X11 forwarding, or agent forwarding.
@@ -63,9 +77,10 @@ the project's mission:
 - ❌ **Bespoke config formats** — configuration is `~/.sshx/settings.json`,
   environment variables, and CLI flags. Nothing else.
 
-**In scope (welcome):** command execution, SFTP file ops, password/secret
-management, named host management, authentication UX, safety checks, and
-cross-platform correctness.
+**In scope (welcome):** agent execution contracts, command execution, SFTP file
+actions, bounded multi-host execution, password/secret references, named host
+management, authentication UX, safety checks, auditing, and cross-platform
+correctness.
 
 ## 4. Architecture
 
@@ -245,9 +260,41 @@ verification for it:
 - Coverage is tracked (Codecov). Coverage is currently modest; **raising it is an
   ongoing goal** — prefer adding tests alongside any change you make.
 
+### Capability coverage matrix (mandatory)
+
+The source of truth is the acceptance matrix in
+[`docs/roadmap.md`](docs/roadmap.md). These requirements are **MUST-level**:
+
+1. Every top-level product capability MUST have at least one Happy Path E2E.
+2. Every high-risk capability MUST cover at least one failure path.
+3. Every permission-sensitive capability MUST verify at least two roles or
+   permission states.
+4. Every state-changing operation MUST verify recovery or rollback after a
+   failure.
+5. Adding a top-level capability MUST include its E2E and an updated matrix row;
+   otherwise the change is incomplete.
+
+Existing unit, component, or local-server tests do not count as CLI E2E unless
+they exercise the compiled `sshx` process across the documented external
+boundary. Record real evidence paths in the matrix and mark missing coverage as
+a gap rather than inferring it.
+
+Canonical commands:
+
+```bash
+make test-short  # unit/component suite without compiled-binary E2E
+make test-e2e    # compiled sshx process across real SSH/SFTP protocol boundaries
+```
+
+The E2E source of evidence is `tests/e2e`. Native OS-keyring lifecycle tests are
+opt-in locally and run against an ephemeral macOS Keychain in CI; never enable
+them against a keyring that cannot be safely isolated and cleaned up.
+
 ## 10. Roadmap
 
-A living, maintainer-adjustable plan. Items must respect the boundaries in §3.
+A living, maintainer-adjustable plan. The authoritative product profile,
+directions, and acceptance matrix are in [`docs/roadmap.md`](docs/roadmap.md).
+Items must respect the boundaries in §3.
 
 **Now / recently shipped**
 

CHANGELOG.md

@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add compiled-binary SSH/SFTP E2E coverage for command execution, structured
+  results, host trust, permissions, partial completion, dry-run, host import,
+  SFTP, server-to-server transfer, keyring-backed sudo, and audit recovery.
+- Run the E2E suite on Linux and macOS CI, including production-binary checks
+  against an ephemeral macOS Keychain.
+
+### Changed
+
+- Upgrade the CI cache and Codecov actions to their supported major versions.
+
+### Fixed
+
+- Make public-key rejection correctly trigger an explicitly configured SSH
+  password fallback; the previous check matched a server-side error type that
+  the SSH client does not return.
+- Keep local audit write failures visible at error log level without changing
+  a successfully completed remote command into a false execution failure.
+
+### Documentation
+
+- Reposition sshx as an agent-native remote host execution tool: SSH is the
+  trusted channel and X is execution. Add the efficiency and security model,
+  hard product boundaries, future directions, and an evidence-based capability
+  coverage matrix with mandatory E2E floors.
+
 ## [0.0.14] - 2026-07-17
 
 ### Added

Makefile

@@ -30,7 +30,7 @@ GOMOD=$(GOCMD) mod
 GOFMT=$(GOCMD) fmt
 GOVET=$(GOCMD) vet
 
-.PHONY: help build build-all test test-verbose test-coverage clean install uninstall run fmt vet lint deps version
+.PHONY: help build build-all test test-short test-e2e test-verbose test-coverage clean install uninstall run fmt vet lint deps version
 
 version: ## Show the version string used for builds
 	@echo "$(VERSION)"
@@ -72,6 +72,10 @@ test-short: ## Run unit tests (skip integration tests)
 	@echo "Running unit tests..."
 	$(GOTEST) -v -short ./...
 
+test-e2e: ## Run compiled-binary SSH/SFTP E2E tests (native keyring is opt-in)
+	@echo "Running compiled-binary E2E tests..."
+	$(GOTEST) -v ./tests/e2e
+
 test-verbose: ## Run verbose tests
 	@echo "Running verbose tests..."
 	$(GOTEST) -v -race ./...

README.md

@@ -11,7 +11,7 @@ $$\   $$ |$$\   $$ |$$ |  $$ |$$  /\$$\
  \______/  \______/ \__|  \__|\__|  \__|
 
 
-Secure SSH & SFTP Client with Built-in Password Manager
+Agent-Native Remote Execution over SSH
 ```
 
 <div align="center">
@@ -44,13 +44,15 @@ English | [简体中文](./README_CN.md)
 
 # SSHX
 
-`sshx` is a barrier-free, cross-platform SSH/SFTP command-line client with a built-in system keyring password manager, making it easy to manage and operate multiple remote servers.
+> **SSH is the channel. X is execution.**
+
+`sshx` is an **agent-native remote host execution tool**. It uses SSH/SFTP to reach existing hosts and brings target resolution, execution preview, safety checks, command and file actions, structured results, and audit evidence into one CLI invocation.
 
 ## Why You Need It?
 
-Managing multiple servers means juggling different passwords and repeatedly entering sudo passwords. `sshx` securely stores passwords in your system keyring and auto-fills sudo passwords, so you can run commands across many servers without the password hassle. One command, multiple servers, zero password hassle.
+Agents do not need another interactive SSH shell. They need a stable, composable remote execution contract with explicit side effects. `sshx` reduces argument assembly through named hosts, removes text guessing through JSON, exit codes, and error kinds, and lowers operational risk through dry-run plans, safety guardrails, the OS keyring, host-key verification, and local auditing.
 
-**New!** Host Configuration Management - Store your frequently used host configurations in `~/.sshx/settings.json` and connect with just a name instead of typing full connection details every time. Each host can have its own SSH private key. Add hosts interactively!
+It remains a single binary with one-shot invocations and no resident component on remote hosts: **efficient, secure, and auditable remote execution for agents over SSH.** Human operators use the same command, preview, and audit semantics for supervision and troubleshooting.
 
 ## Project Structure
 
@@ -60,13 +62,13 @@ Managing multiple servers means juggling different passwords and repeatedly ente
 
 ## Key Features
 
-1. Cross-platform SSH/SFTP operations (supports sudo auto-fill).
-2. Direct server-to-server file transfer (`--transfer=<host>:<path> --to=<host>:<path>`), streamed through the local machine without touching local disk.
-3. Password management (Keychain / Secret Service / Credential Manager).
-4. Host configuration management with per-host SSH keys.
-5. Dry-run execution plan preview for humans and agents.
-6. Local structured audit trail with safe default redaction.
-7. Script execution and command security validation.
+1. Agent-friendly JSON, stable exit codes, separated stdout/stderr, and classified failures.
+2. Dry-run execution plans and default-on local structured auditing with safe redaction.
+3. Named host management and selective OpenSSH config import with per-host SSH keys.
+4. Strict host-key verification, destructive-command guardrails, and explicit bypass semantics.
+5. OS-keyring password management and sudo auto-fill over stdin.
+6. Cross-platform SSH/SFTP command and file actions.
+7. Direct server-to-server transfer, streamed through the local machine without touching local disk.
 
 ## Installation
 
@@ -687,9 +689,14 @@ sudo chmod +x /usr/local/bin/sshx
 
 ## Development
 
+The project's target state, hard non-goals, and capability coverage matrix live in the [Project Profile and Direction](docs/roadmap.md). Every new top-level capability must add a Happy Path E2E and update the matrix; high-risk, permission-sensitive, and state-changing capabilities must also meet the corresponding failure, permission-state, and recovery coverage floors.
+
 ```bash
-# Run tests
-go test ./...
+# Run fast unit/component tests
+make test-short
+
+# Run compiled-binary SSH/SFTP E2E tests
+make test-e2e
 
 # Format code
 gofmt -w .
@@ -703,6 +710,9 @@ make lint
 
 > The lint target requires `golangci-lint` v2.6.1 or newer. Install it with `go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.6.1`.
 
+The normal E2E run uses an isolated, test-only keyring provider. CI additionally
+checks the production binary against an ephemeral macOS Keychain.
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.