[andrewn6]

Runs host-to-host artifact moves as a durable restate service, with the pull pins kept in restate state. Bottom of the restate stack (#346 -> #345 -> #349 -> #348 -> #342).

• Artifacts/{destination snapshot id}.Move: check (a copy already recorded there ends the move), pin, pull, record, release. used by ensureArtifactOnHost (create seeding, off-host fork, migrate) and moveDeltaToHost (migrate deltas, checkpoint chain).
• the pull is a durable.Run step, so it carries the [restate] idempotent host-agent mutations in fc-agent and qemu-agent #338 idempotency key: a retry after a lost response or a restart gets the agent's first answer instead of a second pull. 4xx, immovable and wrong-kind failures are terminal; the rest retry under the ARTIFACTS policy.
• a terminal pull or record failure deletes the partial copy on the destination (unless a record for it exists or cannot be read), then releases the pins.
• a caller that stops waiting without the move's own answer gets ErrArtifactMoveUnsettled; the checkpoint loop then forgets its chain instead of deleting files the move may still use.
• ArtifactPin/{snapshot id} holds pins (holder -> deadline) held until the move releases them on any ending (done, failed, cancelled); only a lazy migrate's seed pin has a delayed-send lapse (35m). artifact gc asks it before evicting and skips anything whose pin it cannot read. the lazy-migrate seed pin moved onto it too; checkpoint chain base pins stay in memory until [restate] snapshot and restore on the Environment object #345.
• the job carries host ids only; agent tokens are read inside the pull step, so none reach the journal.
• ORCH_ARTIFACT_PULL_TIMEOUT_SECONDS is the pull step's timeout on restate, not the request's. the Move handler's abort timeout is set from it.
• metrics: orchestrator_artifact_move_{duration_seconds,bytes_total,failures_total} by source/target host (both modes).
• FUSE_RESTATE_MODE=off keeps the in-process path and in-memory pins unchanged.

no api surface change, so no sdk/openapi changes.

Docs: docs/runbooks/restate.mdx, docs/content/docs/cloud/orchestrator-deploy.mdx (pull timeout row, retry service names), docs/content/docs/cloud/monitoring.mdx.

verified locally

• go build ./..., go vet ./..., gofmt -l ., golangci-lint v2.12.2 (0 issues), go test ./... -count=1, and -race on durable/orchestrator/metrics.
• durable integration tests against a local restate-server (1.6.2 binary; compose pins 1.8.1): move with retried pulls, permanent failure discard, retries running out, already-committed copy, pin lapse/release, and an endpoint restart mid-pull (connections dropped, new endpoint on the same address): pins still held after the restart, the move resumes at the pull and records once.
• docs bun run build passes; bun run lint fails only on pre-existing biome formatting in meta.ts files this PR does not touch.

not run

• the phantom kill -9 mid-move e2e and the gc-across-restart check on real hosts.

Closes #346

[greptile-apps]

Cleanup deletes active move files

runArtifactMove returns an error when its wait ends, but the Restate move keeps running. For checkpoint copies, advanceChain responds by deleting the source checkpoint and retiring the chain, which deletes the destination base without checking durable pins. A dropped connection to Restate or the one-hour wait limit can therefore remove files that the running move still needs, causing the copy to fail. Keep those files until the move finishes or has been cancelled.

[greptile-apps]

Cleanup deletes a recorded copy

ArtifactDiscard treats a failed record lookup as permission to delete the destination copy. If a record write commits but its answer is lost, later write attempts can fail until retries run out. A database read failure during discard then deletes the bytes while leaving the ready record behind. Later creates or moves trust that record and fail when they need the files.

getSnapshotRecord hides database errors as ErrSnapshotNotFound, so this check cannot tell those cases apart. Preserve read errors and retry cleanup unless the record is confirmed absent.

[greptile-apps]

Active move pins expire

Move pins expire 24 hours after acquisition even when the move is still live. The handler acquires them only once, and replay uses those completed calls rather than renewing them. After a long endpoint outage, or with retry delays that span more than a day, Expire can remove the pins before the move finishes. Layer cleanup can then delete its source, leaving the move unable to finish. Renew active move pins, or check whether their move has ended before expiring them.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	fuse	26e448d	Commit Preview URL Branch Preview URL	Nov 15 2026, 06:15 AM