diff --git a/docs/CLI-DESKTOP-HANDOFF.md b/docs/CLI-DESKTOP-HANDOFF.md new file mode 100644 index 0000000..632df29 --- /dev/null +++ b/docs/CLI-DESKTOP-HANDOFF.md @@ -0,0 +1,160 @@ +# Private connector handoff to Desktop (local implementation) + +This same-Mac development path packages a new Collection Profile connector +through the public data-connectors artifact builder. It does not publish, upload, +sign or approve the connector. It requires the matching Desktop implementation; +released Desktop versions without this capability cannot import the handoff. + +Author and check the connector with the existing Factory workflow first: + +```sh +vana connectors build --instructions +vana connectors check --job --state-dir --json +``` + +Use a current public `PDP-Connect/data-connectors` checkout with its locked tools +installed, a real Node 24 executable, and GNU tar. On macOS the CLI discovers +`gtar` and supplies it to the unchanged upstream builder through a temporary PATH +entry. BSD tar is rejected. The command never installs tools or changes the +upstream checkout. + +The locally verified candidate uses dev Account and Moksha on the same Mac in both CLI +and the matching Desktop build. Sign in with `VANA_ENV=dev vana login`. Package a new source name +and the intended streams explicitly. The profile's `connector_key` stays in +kebab case, while the final path segment of `source.id` and selected stream names +must form SDK scopes using lowercase letters, digits and underscores (for example +`desktop_synthetic.items`). The CLI refuses incompatible names without rewriting +them. + +In this example, the checkout directory is `connectors/my_source/`, the manifest +has `connector_key: "my-source"`, and `source.id` ends in `/my_source`. Pass the +directory name `my_source` to the command: the public upstream builder requires +letters, digits and underscores in directory names. + +```sh +VANA_ENV=dev vana connectors desktop my_source \ + --from /absolute/path/to/data-connectors \ + --streams items,profile \ + --node /absolute/path/to/node24 \ + --network moksha --json --no-input +``` + +The result contains `handoffId`, `artifactDigest`, selected streams, exact +`sourceId` and full `scopes` derived from the frozen profile (for example +`desktop_synthetic.items`), Account context, exact `handoffPath`, `openUrl`, and +context-preserving status/resume argv. +Pending approval returns exit 7. JSON, IPC and `--no-input` never open Desktop or +approve anything. For an interactive review add `--open` without those flags, or +open the returned `vana-dev://development-connector?id=...` link (`vana://` for +production). The person reviews the named connector in Sources and chooses Enable and import. +An explicit interactive `desktop-resume --open` returns an already +enabled snapshot to Sources. Its row menu contains details, enable/disable and +remove. Reopening does not activate or import it again. + +```sh +VANA_ENV=dev vana connectors desktop-status --network moksha --json +VANA_ENV=dev vana connectors desktop-resume --network moksha --open +``` + +Status and resume verify every saved payload byte and require the same current +owner, Account authority, service environment and network. They do not rebuild, +activate or recollect. Repeat packaging after edits: unchanged bytes and context +reuse the same ID; changed bytes or context create a new ID. Desktop owns approval, +update activation, disable and removal. An `enabled` result proves only that +Desktop reported this digest enabled; collection, Personal Server storage and +consumer delivery require their own checks. + +## Use the connector in your app + +Keep the app and owner in the same environment/network. Register a local app +with its explicit URL, then complete the returned Account ownership/claim step: + +```sh +VANA_ENV=dev vana app register --network moksha \ + --app-url http://localhost:3090 --app-name "My connector app" --json --no-input +``` + +Use that same app key for its permission request and later reads. Do not create +a replacement app to recover a pending claim. Mainnet availability is a separate +production rollout; this guide does not imply it is ready. + +After the CLI prepares the snapshot, Desktop Sources discovers it for the same Account and network even if no link was opened. The CLI link is a shortcut to the same review: + +1. In the matching Desktop build, open **Sources → Ready to review → Review**, or use the CLI link. Review the imported data and local origin; version, network, scopes and code fingerprint are in Technical details. **Later** leaves it available in Sources without enabling it. +2. Choose **Enable and import** and finish the source's normal setup. Import progress appears in Sources; only this Account and network can run the connector. If enable succeeds but import cannot start, **Retry import** retries collection without repeating activation. +3. Start the Vana request from your consuming app for the exact scope, such as `my_source.items`. Keep Desktop and its registered Personal Server running. Approve the request in Vana, confirm the Account owner-read prompt, and return to the same app/browser session. +4. Read and save the response in the app before acknowledging the request. The approval page returns to the app after acknowledgement. An enabled snapshot alone does not prove that data was imported or delivered. +5. After editing the connector, repeat packaging. Sources shows **Review update** on the existing source row with the active and proposed versions. Choose **Update and import**, then refresh the app through its existing still-valid grant. A public connector release is not required between edits. + +For this release candidate the verified service path is Account dev and Moksha on the same Mac. Use `VANA_ENV=dev` consistently for login, handoff and resume, and select Moksha in Desktop. The CLI release must include `vana connectors desktop --help`; released 0.48.2 alone does not contain this new command. The first public CLI/Desktop version pair must be recorded in the release notes when those artifacts are published. Mainnet release acceptance remains gated on the production Account schema rollout and the fee-eligible server runtime. + +## Recover without replacing your identity + +- Pending approval (exit 7): open the returned link, review and enable the snapshot. `desktop-status` reports Desktop's current receipt. +- Wrong Account or network: select the intended context and repeat status/resume. Do not generate another app key or change Accounts to work around a refusal. +- Changed code: package it again. A changed digest requires explicit approval; the old enabled version remains until the new snapshot is enabled. +- Data unavailable: keep the registered Personal Server running and use the approval page's Try again action. A stale test-app/browser connection is app-owned state; preserve or explicitly clear it there, rather than changing the grant or collecting unrelated sources. +- App return: use the same browser session so the app can recover its saved request. If the app already saved data, retry acknowledgement without repeating collection or the read. + +## Frozen transport + +The shared fixed inbox is `~/.vana/desktop-handoffs/<32-lowercase-hex-id>/`, under +the normal macOS home. `VANA_HOME` still selects CLI authentication state; it does +not relocate this inbox because Desktop must find it independently. Handoffs stay +there for inspection and resume. Do not remove an in-flight handoff. + +`handoff.json` is a strict schema version 1 descriptor with `handoffId`, +`ownerAddress`, canonical `accountUrl`, `environment` (`dev`/`prod`), `network`, +`connectorId`, `version`, selected `streams`, `artifactDigest` and `files`. +Each file record contains a POSIX relative `path`, `sha256` prefixed digest and +byte `size`. Records are sorted by path. The artifact digest is SHA256 of their +UTF-8 concatenation: + +```text +\0\0\n +``` + +`payload/` contains exactly `collection-profile.json`, `code.tar.gz`, +`licenses.tar.gz`, `source-declaration.json`, `provenance.json`, `config.json`, +`layers.json`, and optional `assets.tar.gz`. The upstream tool bundles the shared +runtime; copying the connector source directory would omit required code. +Symlinks, extra payload files, missing layers and changed bytes are refused. +Payload files are limited to 128 MiB each and 256 MiB combined. +The digest establishes integrity, not public review or a new process sandbox. + +Only PATH, normal HOME and temporary-directory location reach the packaging +child. Provider environment values, app keys, browser endpoints, raw captures and +`.env` files are not copied from the checkout. Declare credential environment +**names** in the profile; supply their values through Desktop's existing setup. +The compiled connector and manifest must themselves be safe to distribute: this +command cannot recognize a secret deliberately embedded in source code. + +Desktop derives runtime, setup, source and stream contracts from the verified +payload. It refuses protected-source collisions and unsupported capabilities; +this slice does not replace public connector methods. Standard dev/production +Account authorities are supported; custom Account services are refused. + +The verified Account user, owner address, Account authority and service +environment reserve each private connector and source name on this Mac. +The same Account can repeat packaging with `--network mainnet` after Moksha, +keeping its connector ID and scope. Open the matching network in Desktop and +explicitly approve that network’s snapshot. Each network keeps its own enabled +version, declaration, import progress and setup. Removing an admission retains +its name reservation. Another Account or a second method cannot claim the source. + +`status.json`, written by Desktop beside the descriptor, contains only +`schemaVersion`, `handoffId`, `state` (`pending`, `enabled`, `disabled`, `removed`, +`failed`), `connectorId`, `artifactDigest`, `code` and optional `remedy`. +Missing status means approval is pending. Identity/digest mismatches fail closed. + +No personal provider data, credentials, payment, consent, remote registry, +Mobile distribution or service configuration is part of packaging. + +## Support for this release + +Report CLI/Desktop handoff problems at +https://github.com/vana-com/vana-cli/issues, including CLI/Desktop versions, +source ID, selected network and the visible failure. Keep credentials and raw +provider data out of the report. The tested development route in this guide +is dev Account + Moksha + same-Mac Desktop; Mobile, cross-device code delivery +and public connector submission are outside this first release. diff --git a/src/cli/connector-desktop.ts b/src/cli/connector-desktop.ts new file mode 100644 index 0000000..85a4fb6 --- /dev/null +++ b/src/cli/connector-desktop.ts @@ -0,0 +1,295 @@ +import { execFileSync } from "node:child_process"; +import path from "node:path"; +import type { Command } from "commander"; +import { CliExitCode } from "../core/exit-codes.js"; +import { resolveNetwork } from "../core/network.js"; +import { getVanaHome } from "../core/paths.js"; +import { findLocalConnector } from "../pdpp/local-connectors.js"; +import { loadExportOwner } from "./auth.js"; +import { + assertDesktopHandoffContext, + desktopHandoffInbox, + desktopHandoffScopes, + DesktopHandoffError, + prepareDesktopHandoff, + readDesktopHandoff, + readDesktopHandoffStatus, + type DesktopHandoffContext, +} from "../pdpp/desktop-handoff.js"; + +type Globals = { + json?: boolean; + ipc?: boolean; + noInput?: boolean; + detach?: boolean; + network?: string; +}; +type Options = { + from?: string; + streams?: string; + node?: string; + open?: boolean; +}; +/** External identity and local app transport, injectable without provider or credential access. */ +export interface ConnectorDesktopDeps { + owner: typeof loadExportOwner; + inbox: () => string; + open: (url: string) => void; +} +const defaults: ConnectorDesktopDeps = { + owner: loadExportOwner, + inbox: desktopHandoffInbox, + open: (url) => { + execFileSync("open", [url], { stdio: "ignore" }); + }, +}; +function shellArg(value: string): string { + return /^[a-zA-Z0-9_./:=,-]+$/.test(value) + ? value + : `'${value.replace(/'/g, `'"'"'`)}'`; +} +function action( + command: string, + id: string, + context: DesktopHandoffContext, +): { argv: string[] } { + return { + argv: [ + "env", + `VANA_ENV=${context.environment}`, + `VANA_HOME=${getVanaHome()}`, + "vana", + "connectors", + command, + id, + "--network", + context.network, + "--json", + ], + }; +} + +/** Prepare, inspect or reopen a same-Mac snapshot; approval belongs exclusively to Desktop. */ +export async function runConnectorDesktop( + operation: "prepare" | "status" | "resume", + argument: string, + options: Options, + globals: Globals, + deps: Partial = {}, +): Promise { + const io = { ...defaults, ...deps }; + const machine = globals.json || globals.ipc; + const emit = (output: Record, message: string) => { + process.stdout.write( + machine + ? `${JSON.stringify({ schemaVersion: 1, kind: "desktop_connector_handoff", ...output })}\n` + : `${message}\n`, + ); + }; + try { + if (globals.detach) + throw new DesktopHandoffError( + "unsupported_detach", + "Desktop handoffs do not support --detach.", + ); + if (process.platform !== "darwin") + throw new DesktopHandoffError( + "mac_required", + "This first handoff slice requires Vana Desktop on the same Mac.", + ); + const network = resolveNetwork(globals.network); + const owner = io.owner(); + if (!owner) { + emit( + { + ok: false, + code: "owner_required", + nextAction: { argv: ["vana", "login"] }, + }, + "Sign in to the intended Vana Account with vana login, then repeat this command.", + ); + return CliExitCode.CONFIRMATION_REQUIRED; + } + if (owner.accountUrl !== network.accountUrl) + throw new DesktopHandoffError( + "account_authority_unsupported", + "This Desktop slice supports the standard Account authority for the selected environment. Clear the custom Account URL and sign in there before packaging.", + ); + const context: DesktopHandoffContext = { + ownerAddress: owner.address.toLowerCase(), + accountUrl: owner.accountUrl as DesktopHandoffContext["accountUrl"], + environment: network.env, + network: network.name, + }; + const inbox = io.inbox(); + let descriptor; + if (operation === "prepare") { + const checkout = + options.from ?? (await findLocalConnector(argument))?.path; + if (!checkout) + throw new DesktopHandoffError( + "checkout_required", + "Provide --from /absolute/path/to/data-connectors, or register this local connector first.", + ); + descriptor = await prepareDesktopHandoff({ + checkout, + source: argument, + context, + inbox, + node: options.node, + streams: options.streams?.split(",").map((value) => value.trim()), + }); + } else descriptor = await readDesktopHandoff(inbox, argument); + assertDesktopHandoffContext(descriptor, context); + const { sourceId, scopes } = await desktopHandoffScopes(inbox, descriptor); + const status = await readDesktopHandoffStatus(inbox, descriptor); + const state = status?.state ?? "pending"; + const openUrl = `${context.environment === "dev" ? "vana-dev" : "vana"}://development-connector?id=${descriptor.handoffId}`; + const actions = { + status: action("desktop-status", descriptor.handoffId, context), + resume: action("desktop-resume", descriptor.handoffId, context), + }; + let opened = false; + let openFailed = false; + if ( + operation !== "status" && + options.open && + process.stdout.isTTY && + !globals.noInput && + !machine + ) { + try { + io.open(openUrl); + opened = true; + } catch { + openFailed = true; + } + } + const code = + state === "enabled" + ? "desktop_enabled" + : state === "failed" + ? "desktop_failed" + : state === "removed" + ? "desktop_removed" + : state === "disabled" + ? "desktop_disabled" + : openFailed + ? "desktop_open_failed" + : "desktop_approval_required"; + emit( + { + ok: state === "enabled", + code, + handoffId: descriptor.handoffId, + artifactDigest: descriptor.artifactDigest, + connectorId: descriptor.connectorId, + sourceId, + scopes, + version: descriptor.version, + streams: descriptor.streams, + context, + handoffPath: path.join(inbox, descriptor.handoffId, "handoff.json"), + state, + ...(status ? { desktopCode: status.code } : {}), + openUrl, + opened, + actions, + nextAction: + state === "enabled" ? null : { kind: "desktop_review", url: openUrl }, + }, + `${descriptor.connectorId}@${descriptor.version}: ${state}\nSource: ${sourceId}\nScopes: ${scopes.join(", ")}\nSnapshot: ${descriptor.artifactDigest}\nHandoff: ${path.join(inbox, descriptor.handoffId, "handoff.json")}\n${state === "enabled" ? "Desktop enabled this snapshot. Collection and storage are separate checks." : `Open ${openUrl} in Vana Desktop and review the Development connector. Approval stays in Desktop.`}\nStatus: ${actions.status.argv.map(shellArg).join(" ")}\nResume: ${actions.resume.argv.map(shellArg).join(" ")}`, + ); + return state === "enabled" + ? CliExitCode.OK + : state === "failed" + ? CliExitCode.FAILURE + : CliExitCode.CONFIRMATION_REQUIRED; + } catch (error) { + const safe = error instanceof DesktopHandoffError; + const code = safe ? error.code : "desktop_handoff_failed"; + const message = safe + ? error.message + : "Could not prepare or inspect the handoff. Check the checkout, selected streams and local packaging tools; no Desktop approval was performed."; + emit( + { + ok: false, + code, + message, + nextAction: { argv: ["vana", "connectors", "desktop", "--help"] }, + }, + message, + ); + return [ + "checkout_required", + "invalid_streams", + "unsupported_detach", + "unsupported_layout", + "source_collision", + "invalid_source_scope", + ].includes(code) + ? CliExitCode.USAGE + : CliExitCode.FAILURE; + } +} + +/** Add commands beside Factory jobs and mutable source registration. */ +export function registerConnectorDesktopCommands( + parent: Command, + globals: () => Globals, +): void { + parent + .command("desktop ") + .description( + "Package a frozen private connector for Vana Desktop on this Mac", + ) + .option( + "--from ", + "Public data-connectors checkout containing the connector and artifact builder", + ) + .option( + "--streams ", + "Comma-separated declared streams (default: all)", + ) + .option( + "--node ", + "Real Node 24 executable used by the public artifact builder", + ) + .option( + "--open", + "Open Desktop review on an interactive terminal; never approves it", + ) + .action(async (key: string, options: Options) => { + process.exitCode = await runConnectorDesktop( + "prepare", + key, + options, + globals(), + ); + }); + parent + .command("desktop-status ") + .description( + "Verify a frozen snapshot and inspect its Desktop approval status", + ) + .action(async (id: string) => { + process.exitCode = await runConnectorDesktop("status", id, {}, globals()); + }); + parent + .command("desktop-resume ") + .description( + "Resume the same Desktop handoff without repackaging or duplicate activation", + ) + .option( + "--open", + "Open Desktop review on an interactive terminal; never approves it", + ) + .action(async (id: string, options: Options) => { + process.exitCode = await runConnectorDesktop( + "resume", + id, + options, + globals(), + ); + }); +} diff --git a/src/cli/index.ts b/src/cli/index.ts index 817051b..019457b 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -109,6 +109,7 @@ import { } from "../core/network.js"; import { CliExitCode } from "../core/exit-codes.js"; import { registerAppCommands } from "./app/index.js"; +import { registerConnectorDesktopCommands } from "./connector-desktop.js"; import { registerConnectorBuildCommands } from "./connector-build.js"; import { checkRegisteredServers, @@ -635,6 +636,7 @@ Examples: }); registerConnectorBuildCommands(connectors, () => parsedOptions); + registerConnectorDesktopCommands(connectors, () => parsedOptions); connectors .command("add ") diff --git a/src/pdpp/desktop-handoff.ts b/src/pdpp/desktop-handoff.ts new file mode 100644 index 0000000..6cab8a2 --- /dev/null +++ b/src/pdpp/desktop-handoff.ts @@ -0,0 +1,524 @@ +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { constants } from "node:fs"; +import * as fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { z } from "zod"; +import { readLocalProfile } from "./local.js"; +import { PDPP_PINS } from "./pins.js"; + +const MAX_FILE_BYTES = 128 * 1024 * 1024; +const MAX_TOTAL_BYTES = 256 * 1024 * 1024; +const digestSchema = z.string().regex(/^sha256:[a-f0-9]{64}$/); +const key = z.string().regex(/^[a-z][a-z0-9-]{0,79}$/); +const requiredFiles = [ + "collection-profile.json", + "code.tar.gz", + "licenses.tar.gz", + "source-declaration.json", + "provenance.json", + "config.json", + "layers.json", +]; +const allowedFiles = [...requiredFiles, "assets.tar.gz"]; +const fileSchema = z + .object({ + path: z.enum(allowedFiles as [string, ...string[]]), + sha256: digestSchema, + size: z.number().int().positive().max(MAX_FILE_BYTES), + }) + .strict(); +const descriptorSchema = z + .object({ + schemaVersion: z.literal(1), + handoffId: z.string().regex(/^[a-f0-9]{32}$/), + ownerAddress: z.string().regex(/^0x[a-f0-9]{40}$/), + accountUrl: z.enum([ + "https://account.vana.org", + "https://account-dev.vana.org", + ]), + environment: z.enum(["dev", "prod"]), + network: z.enum(["moksha", "mainnet"]), + connectorId: key, + version: z.string().regex(/^\d+\.\d+\.\d+(?:[-+][a-zA-Z0-9.-]+)?$/), + streams: z + .array(z.string().regex(/^[a-z][a-z0-9_-]{0,79}$/)) + .min(1) + .max(100) + .refine((streams) => new Set(streams).size === streams.length), + artifactDigest: digestSchema, + files: z.array(fileSchema).min(7).max(8), + }) + .strict(); + +/** Immutable local transport; Desktop independently verifies and approves it. */ +export type DesktopHandoff = z.infer; +/** The intended Account authority, owner and chain; never a session token. */ +export type DesktopHandoffContext = Pick< + DesktopHandoff, + "ownerAddress" | "accountUrl" | "environment" | "network" +>; +/** Bounded machine outcomes written by Desktop beside the immutable payload. */ +export const desktopHandoffStatusSchema = z + .object({ + schemaVersion: z.literal(1), + handoffId: z.string().regex(/^[a-f0-9]{32}$/), + state: z.enum(["pending", "enabled", "disabled", "removed", "failed"]), + connectorId: key, + artifactDigest: digestSchema, + code: z.string().regex(/^[a-z][a-z0-9_-]{0,99}$/), + remedy: z.string().max(2000).optional(), + }) + .strict(); + +/** A safe actionable failure; subprocess output is deliberately excluded. */ +export class DesktopHandoffError extends Error { + constructor( + readonly code: string, + message: string, + ) { + super(message); + } +} +const hash = (bytes: string | Buffer) => + createHash("sha256").update(bytes).digest("hex"); +function identity(value: Omit): string { + return hash( + JSON.stringify({ + schemaVersion: value.schemaVersion, + ownerAddress: value.ownerAddress, + accountUrl: value.accountUrl, + environment: value.environment, + network: value.network, + connectorId: value.connectorId, + version: value.version, + streams: value.streams, + artifactDigest: value.artifactDigest, + files: value.files, + }), + ).slice(0, 32); +} +function artifactDigest(files: DesktopHandoff["files"]): string { + return `sha256:${hash(files.map((file) => `${file.path}\0${file.sha256}\0${file.size}\n`).join(""))}`; +} +/** Desktop resolves only this fixed same-Mac inbox, independently of VANA_HOME. */ +export function desktopHandoffInbox(): string { + return path.join(os.homedir(), ".vana", "desktop-handoffs"); +} + +async function regularBytes(file: string): Promise { + const handle = await fs.open(file, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + const stat = await handle.stat(); + if (!stat.isFile() || stat.size > MAX_FILE_BYTES) + throw new Error("invalid file"); + return await handle.readFile(); + } finally { + await handle.close(); + } +} +async function realDirectory(directory: string): Promise { + const stat = await fs.lstat(directory); + if (!stat.isDirectory() || stat.isSymbolicLink()) + throw new Error("invalid directory"); +} + +/** Verify descriptor identity and every frozen byte before status or resume. */ +export async function readDesktopHandoff( + inbox: string, + id: string, +): Promise { + try { + if (!/^[a-f0-9]{32}$/.test(id)) throw new Error("invalid id"); + await realDirectory(inbox); + const directory = path.join(inbox, id); + await realDirectory(directory); + await realDirectory(path.join(directory, "payload")); + const bytes = await regularBytes(path.join(directory, "handoff.json")); + if (bytes.length > 64 * 1024) throw new Error("large descriptor"); + const descriptor = descriptorSchema.parse( + JSON.parse(bytes.toString("utf8")), + ); + const { handoffId, ...content } = descriptor; + if (handoffId !== id || identity(content) !== id) + throw new Error("wrong identity"); + if ( + descriptor.files.reduce((total, file) => total + file.size, 0) > + MAX_TOTAL_BYTES + ) + throw new Error("oversized payload"); + const names = descriptor.files.map((file) => file.path); + if ( + names.join("\0") !== [...new Set(names)].sort().join("\0") || + requiredFiles.some((name) => !names.includes(name)) + ) + throw new Error("invalid inventory"); + if (artifactDigest(descriptor.files) !== descriptor.artifactDigest) + throw new Error("wrong digest"); + const actual = (await fs.readdir(path.join(directory, "payload"))).sort(); + if (actual.join("\0") !== names.join("\0")) throw new Error("extra files"); + for (const file of descriptor.files) { + const data = await regularBytes( + path.join(directory, "payload", file.path), + ); + if (data.length !== file.size || `sha256:${hash(data)}` !== file.sha256) + throw new Error("changed bytes"); + } + return descriptor; + } catch { + throw new DesktopHandoffError( + "handoff_invalid", + "The handoff is missing, malformed or changed. Package the connector again; do not enable this snapshot.", + ); + } +} + +/** Reuse is allowed only in the original Account, owner, service environment and chain. */ +export function assertDesktopHandoffContext( + descriptor: DesktopHandoff, + context: DesktopHandoffContext, +): void { + for (const field of [ + "ownerAddress", + "accountUrl", + "environment", + "network", + ] as const) { + if (descriptor[field] !== context[field]) + throw new DesktopHandoffError( + "handoff_context_changed", + "This handoff belongs to another Account, owner, environment or network. Return to its original context or package a new handoff.", + ); + } +} + +/** Read native status only after checking the immutable handoff and matching identity. */ +export async function readDesktopHandoffStatus( + inbox: string, + descriptor: DesktopHandoff, +): Promise | null> { + try { + const bytes = await regularBytes( + path.join(inbox, descriptor.handoffId, "status.json"), + ); + if (bytes.length > 16 * 1024) throw new Error("large status"); + const status = desktopHandoffStatusSchema.parse( + JSON.parse(bytes.toString("utf8")), + ); + if ( + status.handoffId !== descriptor.handoffId || + status.connectorId !== descriptor.connectorId || + status.artifactDigest !== descriptor.artifactDigest + ) + throw new Error("wrong status identity"); + return status; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; + throw new DesktopHandoffError( + "desktop_status_invalid", + "Desktop returned an invalid status. Reopen the same handoff in Desktop to inspect it.", + ); + } +} + +function toolingEnvironment(): NodeJS.ProcessEnv { + return { + PATH: process.env.PATH ?? "/usr/bin:/bin", + HOME: os.homedir(), + TMPDIR: os.tmpdir(), + }; +} +function commandPath(name: string): string | null { + try { + return execFileSync("which", [name], { + encoding: "utf8", + env: toolingEnvironment(), + stdio: ["ignore", "pipe", "ignore"], + }).trim(); + } catch { + return null; + } +} +function packagingTools(node: string): { node: string; tar: string } { + const executable = path.isAbsolute(node) ? node : commandPath(node); + try { + if ( + !executable || + !/^v24\./.test( + execFileSync(executable, ["--version"], { + encoding: "utf8", + env: toolingEnvironment(), + stdio: ["ignore", "pipe", "ignore"], + }), + ) + ) + throw new Error(); + } catch { + throw new DesktopHandoffError( + "node24_required", + "Packaging needs a real Node 24 executable. Select it on PATH or pass --node /absolute/path/to/node.", + ); + } + for (const name of [ + "gtar", + "tar", + "/opt/homebrew/bin/gtar", + "/usr/local/bin/gtar", + ]) { + const candidate = path.isAbsolute(name) ? name : commandPath(name); + try { + if ( + candidate && + execFileSync(candidate, ["--version"], { + encoding: "utf8", + env: toolingEnvironment(), + stdio: ["ignore", "pipe", "ignore"], + }).includes("GNU tar") + ) + return { node: executable, tar: candidate }; + } catch { + /* Try the next installed implementation. */ + } + } + throw new DesktopHandoffError( + "gnu_tar_required", + "The public artifact builder requires GNU tar. On macOS install gnu-tar and rerun; BSD tar is unsupported.", + ); +} + +function profileScopeSource(profile: unknown): string { + const parsed = z + .object({ source: z.object({ id: z.string() }) }) + .safeParse(profile); + let sourceId: string | undefined; + try { + if (parsed.success) + sourceId = new URL(parsed.data.source.id).pathname + .split("/") + .filter(Boolean) + .at(-1); + } catch { + /* The bounded error below also covers malformed source URIs. */ + } + if (!sourceId || !/^[a-z0-9][a-z0-9_]*$/.test(sourceId)) + throw new DesktopHandoffError( + "invalid_source_scope", + "The profile source.id URI must end in an SDK-compatible source name using lowercase letters, digits and underscores (for example desktop_synthetic). Keep the connector_key in kebab case; no source name is rewritten.", + ); + if (PDPP_PINS.some((pin) => pin.id.replace(/-/g, "_") === sourceId)) + throw new DesktopHandoffError( + "source_collision", + "This source.id belongs to a public source. Choose a new source name; public-source replacement is unsupported.", + ); + return sourceId; +} + +/** Resolve consumer scopes from the verified frozen profile, independently of mutable source code. */ +export async function desktopHandoffScopes( + inbox: string, + descriptor: DesktopHandoff, +): Promise<{ sourceId: string; scopes: string[] }> { + const file = descriptor.files.find( + (entry) => entry.path === "collection-profile.json", + ); + const bytes = await regularBytes( + path.join( + inbox, + descriptor.handoffId, + "payload", + "collection-profile.json", + ), + ); + if ( + !file || + bytes.length !== file.size || + `sha256:${hash(bytes)}` !== file.sha256 + ) + throw new DesktopHandoffError( + "handoff_invalid", + "The frozen profile changed. Do not use scopes from this snapshot.", + ); + const sourceId = profileScopeSource(JSON.parse(bytes.toString("utf8"))); + return { + sourceId, + scopes: descriptor.streams.map((stream) => `${sourceId}.${stream}`), + }; +} + +/** Build through the public upstream tool, then freeze only its explicit payload layers. */ +export async function prepareDesktopHandoff(options: { + checkout: string; + source: string; + streams?: string[]; + context: DesktopHandoffContext; + node?: string; + inbox?: string; +}): Promise { + if (!/^[a-z0-9][a-z0-9_]*$/.test(options.source)) + throw new DesktopHandoffError( + "unsupported_layout", + "Use the upstream connector directory name (lowercase letters, digits and underscores).", + ); + const { files, profile } = await readLocalProfile( + options.checkout, + options.source, + ); + const sourceId = profileScopeSource(profile); + const connectorId = profile.connector_key; + if ( + !key.safeParse(connectorId).success || + PDPP_PINS.some((pin) => pin.id.replace(/_/g, "-") === connectorId) + ) + throw new DesktopHandoffError( + "source_collision", + "Desktop development handoffs require a new source name; public connector replacement is unsupported.", + ); + const streams = [ + ...new Set(options.streams ?? profile.streams.map((stream) => stream.name)), + ].sort(); + if ( + !streams.length || + streams.some( + (stream) => + !/^[a-z0-9][a-z0-9_]*$/.test(stream) || + !profile.streams.some((declared) => declared.name === stream), + ) + ) + throw new DesktopHandoffError( + "invalid_streams", + "Select declared streams with SDK-compatible scope names using lowercase letters, digits and underscores; hyphens are unsupported.", + ); + if ( + !/^[a-z0-9][a-z0-9_]*$/.test(options.source) || + path.dirname(files.entry) !== + path.join(files.root, "connectors", options.source) + ) + throw new DesktopHandoffError( + "unsupported_layout", + "Packaging requires the current upstream connectors//manifest.json layout.", + ); + const builder = path.join( + files.root, + "scripts", + "build-connector-oci-artifact.mjs", + ); + await fs.access(builder).catch(() => { + throw new DesktopHandoffError( + "builder_missing", + "Use a public data-connectors checkout containing scripts/build-connector-oci-artifact.mjs and its locked installed tools.", + ); + }); + const tools = packagingTools(options.node ?? "node"); + const inbox = options.inbox ?? desktopHandoffInbox(); + await fs.mkdir(inbox, { recursive: true, mode: 0o700 }); + await realDirectory(inbox); + const temporary = await fs.mkdtemp(path.join(inbox, ".prepare-")); + try { + const toolBin = path.join(temporary, "bin"); + await fs.mkdir(toolBin, { mode: 0o700 }); + await fs.symlink(tools.tar, path.join(toolBin, "tar")); + const payload = path.join(temporary, "payload"); + try { + execFileSync( + tools.node, + [builder, "--connector", options.source, "--out", payload], + { + cwd: files.root, + stdio: ["ignore", "pipe", "pipe"], + maxBuffer: 1024 * 1024, + // A build must not inherit provider secrets, PDPP browser endpoints or collection inputs. + env: { + PATH: `${toolBin}${path.delimiter}${process.env.PATH ?? "/usr/bin:/bin"}`, + HOME: os.homedir(), + TMPDIR: os.tmpdir(), + }, + }, + ); + } catch { + throw new DesktopHandoffError( + "packaging_failed", + "The public artifact builder failed. Run its static checks locally and inspect its private diagnostics; no handoff was created.", + ); + } + const names = (await fs.readdir(payload)).sort(); + if ( + names.some((name) => !allowedFiles.includes(name)) || + requiredFiles.some((name) => !names.includes(name)) + ) + throw new DesktopHandoffError( + "payload_invalid", + "The builder output is incomplete or includes unsupported files. No handoff was created.", + ); + const inventory: DesktopHandoff["files"] = []; + for (const name of names) { + const data = await regularBytes(path.join(payload, name)); + await fs.chmod(path.join(payload, name), 0o600); + inventory.push({ + path: name, + sha256: `sha256:${hash(data)}`, + size: data.length, + }); + } + if ( + inventory.reduce((total, file) => total + file.size, 0) > MAX_TOTAL_BYTES + ) + throw new DesktopHandoffError( + "payload_too_large", + "The Desktop payload exceeds the 256 MiB total limit.", + ); + const frozenProfile = JSON.parse( + ( + await regularBytes(path.join(payload, "collection-profile.json")) + ).toString("utf8"), + ); + if ( + profileScopeSource(frozenProfile) !== sourceId || + frozenProfile.connector_key !== connectorId || + frozenProfile.version !== profile.version || + streams.some( + (stream) => + !frozenProfile.streams?.some( + (entry: { name: string }) => entry.name === stream, + ), + ) + ) + throw new DesktopHandoffError( + "source_changed", + "The manifest changed during packaging. Rerun packaging after edits finish.", + ); + const content = { + schemaVersion: 1 as const, + ...options.context, + connectorId, + version: profile.version, + streams, + artifactDigest: artifactDigest(inventory), + files: inventory, + }; + const descriptor = descriptorSchema.parse({ + ...content, + handoffId: identity(content), + }); + await fs.writeFile( + path.join(temporary, "handoff.json"), + `${JSON.stringify(descriptor, null, 2)}\n`, + { mode: 0o600 }, + ); + await fs.rm(toolBin, { recursive: true }); + const destination = path.join(inbox, descriptor.handoffId); + try { + await fs.rename(temporary, destination); + } catch (error) { + if ( + !["EEXIST", "ENOTEMPTY"].includes( + (error as NodeJS.ErrnoException).code ?? "", + ) + ) + throw error; + } + return await readDesktopHandoff(inbox, descriptor.handoffId); + } finally { + await fs.rm(temporary, { recursive: true, force: true }); + } +} diff --git a/test/cli/connector-desktop.test.ts b/test/cli/connector-desktop.test.ts new file mode 100644 index 0000000..91d9359 --- /dev/null +++ b/test/cli/connector-desktop.test.ts @@ -0,0 +1,53 @@ +import { Command } from "commander"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { + registerConnectorDesktopCommands, + runConnectorDesktop, +} from "../../src/cli/connector-desktop.js"; + +afterEach(() => { + vi.restoreAllMocks(); +}); +describe("connectors desktop command", () => { + it("keeps frozen handoff syntax separate from Factory resume and accepts packaging options", async () => { + const program = new Command().exitOverride(); + registerConnectorDesktopCommands(program.command("connectors"), () => ({})); + const parent = program.commands[0]; + const desktop = parent.commands.find( + (command) => command.name() === "desktop", + )!; + expect(desktop.helpInformation()).toContain("--from "); + expect(desktop.helpInformation()).toContain("--streams "); + expect(desktop.helpInformation()).toContain("--node "); + expect(parent.commands.map((command) => command.name())).toEqual([ + "desktop", + "desktop-status", + "desktop-resume", + ]); + }); + + it.skipIf(process.platform !== "darwin")( + "returns a structured sign-in action without prompting or opening Desktop", + async () => { + let output = ""; + vi.spyOn(process.stdout, "write").mockImplementation((value) => { + output += value; + return true; + }); + const open = vi.fn(); + const exit = await runConnectorDesktop( + "prepare", + "desktop_synthetic", + { open: true }, + { json: true, noInput: true }, + { owner: () => null, open }, + ); + expect(exit).toBe(7); + expect(JSON.parse(output)).toMatchObject({ + code: "owner_required", + nextAction: { argv: ["vana", "login"] }, + }); + expect(open).not.toHaveBeenCalled(); + }, + ); +}); diff --git a/test/pdpp/desktop-handoff.test.ts b/test/pdpp/desktop-handoff.test.ts new file mode 100644 index 0000000..6c564e9 --- /dev/null +++ b/test/pdpp/desktop-handoff.test.ts @@ -0,0 +1,362 @@ +import { + chmod, + mkdir, + mkdtemp, + readFile, + readdir, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { runConnectorDesktop } from "../../src/cli/connector-desktop.js"; +import { + assertDesktopHandoffContext, + desktopHandoffScopes, + prepareDesktopHandoff, + readDesktopHandoff, + readDesktopHandoffStatus, +} from "../../src/pdpp/desktop-handoff.js"; + +let directory: string; +let checkout: string; +let inbox: string; +let node: string; +let prepared: Awaited>; +const context = { + ownerAddress: `0x${"1".repeat(40)}`, + accountUrl: "https://account-dev.vana.org" as const, + environment: "dev" as const, + network: "moksha" as const, +}; +const profile = { + connector_key: "desktop-synthetic", + source: { id: "https://registry.pdpp.dev/sources/desktop_synthetic" }, + version: "0.0.1", + streams: [{ name: "items" }, { name: "profile" }], +}; +async function prepare(streams?: string[]) { + return prepareDesktopHandoff({ + checkout, + source: "desktop_synthetic", + context, + inbox, + node, + streams, + }); +} +// This external builder fixture exercises transport/storage, not upstream archive compatibility. +const builder = `import * as fs from "node:fs"; +import path from "node:path"; +if (process.env.VANA_APP_PRIVATE_KEY || process.env.PDPP_BROWSER_ENDPOINT || process.env.EXAMPLE_API_TOKEN) throw Error("secret environment inherited"); +const out=process.argv[process.argv.indexOf("--out")+1]; +fs.mkdirSync(out,{recursive:true}); +fs.copyFileSync("connectors/desktop_synthetic/manifest.json",path.join(out,"collection-profile.json")); +fs.copyFileSync("connectors/desktop_synthetic/index.ts",path.join(out,"code.tar.gz")); +for (const name of ["licenses.tar.gz","source-declaration.json","provenance.json","config.json","layers.json"]) fs.writeFileSync(path.join(out,name),"synthetic-fixture"); +`; +beforeEach(async () => { + directory = await mkdtemp(path.join(os.tmpdir(), "vana-handoff-test-")); + checkout = path.join(directory, "checkout with spaces"); + inbox = path.join(directory, "inbox"); + await mkdir(path.join(checkout, "connectors", "desktop_synthetic"), { + recursive: true, + }); + await mkdir(path.join(checkout, "scripts")); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", "manifest.json"), + JSON.stringify(profile), + ); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", "index.ts"), + "export const synthetic = 'v1';", + ); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", ".env"), + "secret-env-sentinel", + ); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", "capture.json"), + "raw-capture-sentinel", + ); + await writeFile( + path.join(checkout, "scripts", "build-connector-oci-artifact.mjs"), + builder, + ); + // The subprocess is synthetic; the separate real upstream smoke proves actual Node 24/GNU tar packaging. + node = path.join(directory, "fixture-node"); + await writeFile( + node, + `#!/bin/sh\nif [ "$1" = "--version" ]; then echo v24.22.1; else exec '${process.execPath.replace(/'/g, `'"'"'`)}' "$@"; fi\n`, + ); + await chmod(node, 0o700); + prepared = await prepare(["items"]); +}); +afterEach(async () => { + vi.unstubAllEnvs(); + await rm(directory, { recursive: true, force: true }); +}); + +describe("Desktop frozen handoff", () => { + it("reuses the immutable identity, excludes credentials/captures, and makes changed code a new snapshot", async () => { + vi.stubEnv("VANA_APP_PRIVATE_KEY", "key-sentinel"); + vi.stubEnv("PDPP_BROWSER_ENDPOINT", "private-browser-sentinel"); + vi.stubEnv("EXAMPLE_API_TOKEN", "token-sentinel"); + const first = prepared; + const again = await prepare(["items"]); + expect(again.handoffId).toBe(first.handoffId); + expect(again.artifactDigest).toBe(first.artifactDigest); + expect(await readdir(inbox)).toEqual([first.handoffId]); + const payload = path.join(inbox, first.handoffId, "payload"); + expect((await readdir(payload)).sort()).toEqual([ + "code.tar.gz", + "collection-profile.json", + "config.json", + "layers.json", + "licenses.tar.gz", + "provenance.json", + "source-declaration.json", + ]); + const saved = await readFile( + path.join(inbox, first.handoffId, "handoff.json"), + "utf8", + ); + expect(saved).not.toMatch( + /key-sentinel|private-browser-sentinel|token-sentinel|secret-env-sentinel|raw-capture-sentinel/, + ); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", "index.ts"), + "export const synthetic = 'v2';", + ); + const changed = await prepare(["items"]); + expect(changed.handoffId).not.toBe(first.handoffId); + expect(changed.artifactDigest).not.toBe(first.artifactDigest); + expect(await readFile(path.join(payload, "code.tar.gz"), "utf8")).toBe( + "export const synthetic = 'v1';", + ); + }); + + it("rejects changed bytes, extra files and symlink replacement before resume", async () => { + const handoff = prepared; + const payload = path.join(inbox, handoff.handoffId, "payload"); + const code = path.join(payload, "code.tar.gz"); + const original = await readFile(code); + await writeFile(code, "tampered"); + await expect( + readDesktopHandoff(inbox, handoff.handoffId), + ).rejects.toMatchObject({ code: "handoff_invalid" }); + await writeFile(code, original); + await writeFile(path.join(payload, ".env"), "extra"); + await expect( + readDesktopHandoff(inbox, handoff.handoffId), + ).rejects.toMatchObject({ code: "handoff_invalid" }); + await rm(path.join(payload, ".env")); + const replacement = path.join(directory, "replacement"); + await writeFile(replacement, original); + await rm(code); + await symlink(replacement, code); + await expect( + readDesktopHandoff(inbox, handoff.handoffId), + ).rejects.toMatchObject({ code: "handoff_invalid" }); + }); + + it("binds status/resume to owner, Account authority, environment, network and artifact", async () => { + const handoff = prepared; + expect(() => assertDesktopHandoffContext(handoff, context)).not.toThrow(); + for (const changed of [ + { ownerAddress: `0x${"2".repeat(40)}` }, + { accountUrl: "https://account.vana.org" as const }, + { environment: "prod" as const }, + { network: "mainnet" as const }, + ]) { + expect(() => + assertDesktopHandoffContext(handoff, { ...context, ...changed }), + ).toThrow(/another Account/); + } + expect(await readDesktopHandoffStatus(inbox, handoff)).toBeNull(); + await writeFile( + path.join(checkout, "connectors", "desktop_synthetic", "manifest.json"), + JSON.stringify({ + ...profile, + source: { id: "https://registry.pdpp.dev/sources/changed_checkout" }, + }), + ); + expect(await desktopHandoffScopes(inbox, handoff)).toEqual({ + sourceId: "desktop_synthetic", + scopes: ["desktop_synthetic.items"], + }); + if (process.platform === "darwin") { + vi.stubEnv("VANA_ENV", "dev"); + let output = ""; + const write = vi + .spyOn(process.stdout, "write") + .mockImplementation((value) => { + output += value; + return true; + }); + const open = vi.fn(); + const exit = await runConnectorDesktop( + "resume", + handoff.handoffId, + { open: true }, + { json: true, noInput: true, network: "moksha" }, + { + owner: () => ({ + address: context.ownerAddress, + accountUrl: context.accountUrl, + }), + inbox: () => inbox, + open, + }, + ); + write.mockRestore(); + expect(exit).toBe(7); + expect(open).not.toHaveBeenCalled(); + expect(JSON.parse(output)).toMatchObject({ + handoffId: handoff.handoffId, + state: "pending", + connectorId: "desktop-synthetic", + sourceId: "desktop_synthetic", + scopes: ["desktop_synthetic.items"], + artifactDigest: handoff.artifactDigest, + opened: false, + openUrl: `vana-dev://development-connector?id=${handoff.handoffId}`, + }); + } + const statusFile = path.join(inbox, handoff.handoffId, "status.json"); + const status = { + schemaVersion: 1, + handoffId: handoff.handoffId, + connectorId: handoff.connectorId, + artifactDigest: handoff.artifactDigest, + state: "enabled", + code: "enabled", + }; + await writeFile(statusFile, JSON.stringify(status)); + expect(await readDesktopHandoffStatus(inbox, handoff)).toEqual(status); + await writeFile( + statusFile, + JSON.stringify({ ...status, artifactDigest: `sha256:${"0".repeat(64)}` }), + ); + await expect( + readDesktopHandoffStatus(inbox, handoff), + ).rejects.toMatchObject({ code: "desktop_status_invalid" }); + }); + + it.skipIf(process.platform !== "darwin")( + "explicitly reopens an enabled snapshot for management while automated modes stay closed", + async () => { + vi.stubEnv("VANA_ENV", "dev"); + const status = { + schemaVersion: 1, + handoffId: prepared.handoffId, + connectorId: prepared.connectorId, + artifactDigest: prepared.artifactDigest, + state: "enabled", + code: "enabled", + }; + await writeFile( + path.join(inbox, prepared.handoffId, "status.json"), + JSON.stringify(status), + ); + const previousTTY = Object.getOwnPropertyDescriptor( + process.stdout, + "isTTY", + ); + Object.defineProperty(process.stdout, "isTTY", { + value: true, + configurable: true, + }); + const output = vi + .spyOn(process.stdout, "write") + .mockImplementation(() => true); + const open = vi.fn(); + const deps = { + owner: () => ({ + address: context.ownerAddress, + accountUrl: context.accountUrl, + }), + inbox: () => inbox, + open, + }; + try { + expect( + await runConnectorDesktop( + "resume", + prepared.handoffId, + { open: true }, + { network: "moksha" }, + deps, + ), + ).toBe(0); + expect(open).toHaveBeenCalledExactlyOnceWith( + `vana-dev://development-connector?id=${prepared.handoffId}`, + ); + await runConnectorDesktop( + "resume", + prepared.handoffId, + { open: true }, + { network: "moksha", noInput: true }, + deps, + ); + await runConnectorDesktop( + "resume", + prepared.handoffId, + { open: true }, + { network: "moksha", json: true }, + deps, + ); + expect(open).toHaveBeenCalledTimes(1); + expect(await readDesktopHandoffStatus(inbox, prepared)).toEqual(status); + } finally { + output.mockRestore(); + if (previousTTY) + Object.defineProperty(process.stdout, "isTTY", previousTTY); + else delete process.stdout.isTTY; + } + }, + ); + + it("refuses scope-incompatible source/stream names and public source collisions before packaging", async () => { + inbox = path.join(directory, "scope-incompatible-inbox"); + const manifest = path.join( + checkout, + "connectors", + "desktop_synthetic", + "manifest.json", + ); + for (const [source, code] of [ + ["desktop-synthetic", "invalid_source_scope"], + ["spotify", "source_collision"], + ]) { + await writeFile( + manifest, + JSON.stringify({ + ...profile, + source: { id: `https://registry.pdpp.dev/sources/${source}` }, + }), + ); + await expect(prepare()).rejects.toMatchObject({ code }); + } + await writeFile( + manifest, + JSON.stringify({ ...profile, streams: [{ name: "bad-stream" }] }), + ); + await expect(prepare()).rejects.toMatchObject({ code: "invalid_streams" }); + }); + + it("refuses undeclared streams and incomplete builder output with no persisted handoff", async () => { + inbox = path.join(directory, "invalid-inbox"); + await expect(prepare(["missing"])).rejects.toMatchObject({ + code: "invalid_streams", + }); + await writeFile( + path.join(checkout, "scripts", "build-connector-oci-artifact.mjs"), + builder + 'fs.writeFileSync(path.join(out,".env"),"never ship");', + ); + await expect(prepare()).rejects.toMatchObject({ code: "payload_invalid" }); + expect(await readdir(inbox)).toEqual([]); + }); +});