Sync FS operations

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>

Author: alexellis

docs/reference/api.md

@@ -176,6 +176,30 @@ Add a host with a custom GitHub user to override the SSH keys:
 }
 ```
 
+### Block until ready (`?wait=` and `?timeout=`)
+
+By default, `POST /hostgroup/NAME/nodes` returns as soon as the VM is
+scheduled — the in-guest agent may still be booting. Two optional query
+parameters let the server hold the response until readiness, so callers
+don't need their own poll loop on `/vm/{hostname}/health`:
+
+- `wait=agent` — block until `agent_uptime > 0` (the agent is up and can
+  accept `exec` calls).
+- `wait=userdata` — additionally block until `userdata_ran == true`
+  (the cloud-init-style userdata script has completed).
+- `timeout=30s` — Go-style duration. Defaults to `5m` when `wait` is
+  set. On timeout the VM is cleaned up and the endpoint returns **408**.
+
+```bash
+curl --unix-socket ~/slicer-mac/slicer.sock \
+  -X POST "http://localhost/hostgroup/sbox/nodes?wait=agent&timeout=30s" \
+  -H "Content-Type: application/json" \
+  -d '{"cpus":1,"ram_bytes":1073741824}'
+```
+
+Typical response latency on Slicer-for-Mac is ~450 ms for `wait=agent`
+on a 1 vCPU / 1 GiB sandbox.
+
 ## List nodes in a Host Group
 
 HTTP GET
@@ -366,11 +390,18 @@ Query parameters:
 - `stdout` (optional): enable stdout capture (true/false)
 - `stderr` (optional): enable stderr capture (true/false)
 - `permissions` (optional): permissions for the command
+- `buffered` (optional): see **Buffered exec** below.
 
 Body: stdin data (if `stdin=true`)
 
 Response: streaming newline-delimited JSON with stdout/stderr/exit_code
 
+Newer agents emit typed frames on the streaming response — `started`
+(first frame, carries `pid` and `started_at`), `stdout`, `stderr`, and
+`exit` (last frame, carries `exit_code` and `ended_at`). Older agents
+emit untyped frames that mix `stdout`/`stderr`/`exit_code` on a single
+object; clients should tolerate both shapes.
+
 Example response stream:
 
 ```json
@@ -389,6 +420,87 @@ The `error` variable may also be populated if there was a problem running, start
 {"error": "Some error message", "exit_code": 1}
 ```
 
+### Buffered exec (`?buffered=true`)
+
+For callers that don't need live output, add `buffered=true` to get a
+single JSON response instead of an NDJSON stream:
+
+```bash
+curl --unix-socket ~/slicer-mac/slicer.sock \
+  -X POST "http://localhost/vm/sbox-1/exec?buffered=true&cmd=/bin/bash&args=-lc&args=echo%20hello" \
+  -H "Content-Type: application/json"
+```
+
+Response:
+
+```json
+{ "stdout": "hello\n", "stderr": "", "exit_code": 0 }
+```
+
+The request shape is the same (query parameters, not a JSON body). The
+response does not include `started_at` / `ended_at` — those are only on
+streaming frames. `stdin` is not supported with `buffered=true`; use
+the streaming endpoint if you need to pipe stdin.
+
+## Filesystem operations
+
+HTTP endpoints under `/vm/{hostname}/fs/*` read and mutate the guest
+filesystem directly via the in-guest agent. Prefer these over shelling
+out through `/exec` — they're faster, don't depend on `ls`/`stat`/`mkdir`
+being on the guest PATH, and avoid shell-quoting hazards on unusual
+filenames.
+
+### List a directory
+
+HTTP GET
+
+`/vm/{hostname}/fs/readdir?path=/etc`
+
+Response:
+
+```json
+[
+  {"name":"hostname","type":"file","size":7,"mtime":1776024825,"mode":"0644"},
+  {"name":"ssh","type":"directory","size":4096,"mtime":1776024825,"mode":"0755"},
+  {"name":"localtime","type":"symlink","size":27,"mtime":1775348015,"mode":"0777"}
+]
+```
+
+`type` is one of `file`, `directory`, or `symlink`. `mtime` is a Unix
+timestamp in seconds; `mode` is the octal file mode.
+
+### Stat a single entry
+
+HTTP GET
+
+`/vm/{hostname}/fs/stat?path=/etc/hostname`
+
+Response: a single `SlicerFSInfo` object (same shape as one entry from
+`readdir`). Returns **404** if the path does not exist — useful as a
+cheap `exists` check.
+
+### Create a directory
+
+HTTP POST
+
+`/vm/{hostname}/fs/mkdir`
+
+Body:
+
+```json
+{ "path": "/tmp/work", "recursive": true, "mode": "0755" }
+```
+
+`recursive` and `mode` are optional.
+
+### Remove a file or directory
+
+HTTP DELETE
+
+`/vm/{hostname}/fs/remove?path=/tmp/work&recursive=true`
+
+Returns 200 on success.
+
 ## Manage secrets
 
 HTTP GET
@@ -497,6 +609,50 @@ HTTP POST
 
 Response 200: text/plain
 
+## Suspend a VM to disk (Slicer-for-Mac only, for now)
+
+HTTP POST
+
+`/vm/{hostname}/suspend`
+
+Takes a Firecracker snapshot of the VM's memory and disk state, then
+shuts down the underlying VM process. Memory is released. Use
+`/restore` to bring the VM back up from the snapshot.
+
+> Availability: currently supported on Slicer-for-Mac only. Not
+> available on the Linux daemon yet.
+
+Response 200: text/plain
+
+## Restore a VM from a snapshot (Slicer-for-Mac only, for now)
+
+HTTP POST
+
+`/vm/{hostname}/restore`
+
+Restores a previously-suspended VM from its Firecracker snapshot. The
+VM resumes with its memory and disk state intact.
+
+> Availability: currently supported on Slicer-for-Mac only. Not
+> available on the Linux daemon yet.
+
+Response 200: text/plain
+
+## Relaunch an API-launched VM
+
+HTTP POST
+
+`/vm/{hostname}/relaunch`
+
+Bring a previously-created API-launched VM back up using its existing
+disk image, without going through a fresh `POST /hostgroup/NAME/nodes`.
+Useful after an intentional shutdown when you want to resume the same
+sandbox identity rather than spin up a new one.
+
+Available on both Slicer-for-Mac and the Linux daemon.
+
+Response 200: `SlicerCreateNodeResponse`
+
 ## Forward TCP connections
 
 HTTP GET

docs/tasks/execute-commands.md

@@ -45,3 +45,30 @@ Combine with local commands using pipes and STDIO:
 # Pipe local file content to VM command
 cat /etc/hostname | slicer vm exec vm-1 -- base64 --wrap 9999
 ```
+
+## Streaming vs buffered exec via the REST API
+
+The `/vm/{hostname}/exec` REST endpoint ships two response shapes:
+
+- **Streaming** (default) — NDJSON frames as output arrives. Preferred
+  for long-running commands where you want live stdout/stderr, or when
+  you want to measure process-start latency separately from first-byte
+  latency using the typed `started` frame.
+- **Buffered** — add `buffered=true` to get a single JSON document with
+  `stdout`, `stderr`, and `exit_code` once the process exits. Preferred
+  for short "run one thing, give me the result" calls; avoids the need
+  for client-side NDJSON parsing.
+
+```bash
+# Streaming (default)
+curl --unix-socket ~/slicer-mac/slicer.sock \
+  -X POST "http://localhost/vm/vm-1/exec?cmd=uname&args=-a"
+
+# Buffered
+curl --unix-socket ~/slicer-mac/slicer.sock \
+  -X POST "http://localhost/vm/vm-1/exec?buffered=true&cmd=uname&args=-a"
+# → {"stdout":"Linux vm-1 ...\n","stderr":"","exit_code":0}
+```
+
+`buffered=true` does not accept `stdin`; use the streaming endpoint if
+you need to pipe stdin data.

openapi.yaml

@@ -311,10 +311,48 @@ components:
 
     SlicerExecWriteResult:
       type: object
+      description: |
+        One NDJSON frame from a streaming exec. Typed exec frames add
+        `started` (first frame, emitted when the process is spawned) and
+        `exit` (last frame, carries exit_code). Intermediate frames carry
+        stdout or stderr chunks. Older agents may emit untyped frames that
+        mix stdout/stderr/exit_code; clients should tolerate both shapes.
       properties:
+        type:
+          type: string
+          enum: [started, stdout, stderr, exit]
+          description: Frame type (newer agents). Absent on older agents.
         timestamp:
           type: string
           format: date-time
+        started_at:
+          type: string
+          format: date-time
+          description: Present on the `started` frame only.
+        ended_at:
+          type: string
+          format: date-time
+          description: Present on the `exit` frame only.
+        pid:
+          type: integer
+          description: Present on the `started` frame only.
+        stdout:
+          type: string
+        stderr:
+          type: string
+        exit_code:
+          type: integer
+        error:
+          type: string
+
+    ExecResult:
+      type: object
+      description: |
+        Buffered exec response — returned when the caller passes
+        `?buffered=true` to `POST /vm/{hostname}/exec`. A single JSON
+        document instead of an NDJSON stream.
+      required: [stdout, stderr, exit_code]
+      properties:
         stdout:
           type: string
         stderr:
@@ -323,6 +361,41 @@ components:
           type: integer
         error:
           type: string
+          description: Non-empty when the agent could not run the command.
+
+    SlicerFSInfo:
+      type: object
+      description: Filesystem entry metadata returned by `/fs/readdir` and `/fs/stat`.
+      required: [name, type, size, mtime, mode]
+      properties:
+        name:
+          type: string
+        type:
+          type: string
+          enum: [file, directory, symlink]
+        size:
+          type: integer
+          format: int64
+        mtime:
+          type: integer
+          format: int64
+          description: Modification time as a Unix timestamp (seconds).
+        mode:
+          type: string
+          description: Octal file mode, zero-padded (e.g. "0644").
+
+    SlicerFSMkdirRequest:
+      type: object
+      required: [path]
+      properties:
+        path:
+          type: string
+        recursive:
+          type: boolean
+          default: false
+        mode:
+          type: string
+          description: Octal mode for the created directory (e.g. "0755").
 
     SlicerAgentHealthResponse:
       type: object
@@ -420,13 +493,37 @@ paths:
 
     post:
       summary: Create a node in a host group
+      description: |
+        Optionally block the response until the in-guest agent reports ready
+        (`wait=agent`) or until cloud-init-style userdata finishes
+        (`wait=userdata`). Server-side wait removes the need for client-side
+        `/health` polling and returns a 408 on timeout (the VM is cleaned up
+        before the error is returned).
       security:
         - BearerAuth: []
       parameters:
         - in: path
           name: name
           required: true
           schema: { type: string }
+        - in: query
+          name: wait
+          description: |
+            Block the response until readiness. `agent` waits for agent_uptime
+            > 0 (process is ready to receive exec). `userdata` additionally
+            waits for userdata_ran == true. Omit to return as soon as the VM
+            is scheduled.
+          schema:
+            type: string
+            enum: [agent, userdata]
+        - in: query
+          name: timeout
+          description: |
+            Go-style duration (e.g. `30s`, `2m`). Only meaningful when `wait`
+            is set. Defaults to `5m`.
+          schema:
+            type: string
+            default: "5m"
       requestBody:
         required: true
         content:
@@ -458,6 +555,11 @@ paths:
           content:
             application/json:
               schema: { $ref: '#/components/schemas/ErrorResponse' }
+        "408":
+          description: Wait timed out before readiness; VM is cleaned up
+          content:
+            application/json:
+              schema: { $ref: '#/components/schemas/ErrorResponse' }
         "500":
           description: Internal error
           content: