Repository navigation
Map gateway, Node, Go runner, payment, and management boundaries #99
Description
Outcome: The gateway, Node, runner, payment and management boundaries are mapped from pinned source, and the selected k3s/gVisor topology and remaining delivery questions have owners. The allocator owns orchestrator pod upgrades; ordinary HTTP app images need no Livepeer awareness.
Parent: #82. Reviewed source references: Node 7a75438ed94fa0472bb6c70efe5f81f5193ab728; Go runner 38b527c7e74b3801e5ddd67e51ad52d98576bfe0. These are evidence for the mapping, not sandbox release pins or a deployed-cluster test.
Current protocol boundaries
| Boundary | Implemented behavior |
|---|---|
| Discovery → client | Node /discovery, standalone aggregation and signer discovery advertise orchestrator-owned app URLs, capacity, mode, pricing and optional GPU metadata. Discovery is inventory, not placement or hardware qualification. |
| Client → orchestrator | Persistent mode reserves through POST /apps/{runner_id}/session and uses the returned app_url; single-shot mode uses /apps/{runner_id}/app. Node proxies HTTP/SSE/WebSocket and Trickle. |
| Orchestrator ↔ runner | Dynamic Go SDK registration uses a bootstrap credential, then a runner-scoped heartbeat credential. Node injects session ID/token/control headers and sends reserved/released O2R events; callbacks authorize runner controls. Static runners are also supported but RunnerConfig loads at startup. |
| Payment/redemption | Clients provide signed tickets. Node currently validates/credits/charges, durably stores winning tickets, and runs redemption with a keystore in the orchestrator process. Session and payment control state are process-local; winning tickets persist. |
| Management | Registration, unregister and session stop are protocol controls. Neither reviewed repo implements allocator enrollment, Kubernetes reconciliation, arbitrary deployment, approved exec, host upgrades or delegated management authorization. |
Evidence: Node route/auth contract, orchestrator HTTP/proxy implementation, Go registration/O2R, payment recovery/storage.
Selected target topology (decision recorded 2026-10-29; implementation pending)
- One k3s cluster per machine. The allocator controls and upgrades the orchestrator pod through locally enforced delegation (Define sandbox delegation, wallet-secret boundaries, and management topology #83/Enforce delegated sandbox permissions and protect operator secrets locally #85/Implement delegated node management and orchestrator lifecycle with draining #94). The management transport remains to be selected (Prototype authenticated outbound management connectivity for NATed nodes #95).
- Run existing HTTP application images in separate gVisor pods. App images contain no Livepeer code or runner bootstrap credential. App-originated egress cannot reach any k3s endpoint; orchestrator-to-app HTTP ingress is allowed (Enforce delegated sandbox permissions and protect operator secrets locally #85/Deploy ordinary HTTP app images in gVisor and register them with Node #92/Verify sandboxed HTTP app execution through Node and the Python SDK #93). A trusted k3s adapter pod speaks Node's heartbeat protocol directly. Runtime and adapter code live in a new repository. The current protocol supports one capability per runner identity during development; livepeer/node #25 will provide multi-capability registration before launch. The adapter holds runner credentials and reconciles registration, readiness, capacity, session events and unregister; it does not proxy app requests. The sandbox path has no Go SDK dependency.
- App configuration supports reusable instances shared across calls or sessions and capacity-one single-session instances dedicated to a persistent session or single-shot call. Several instances of the same app may run concurrently. Each ready instance can register the same capability under a distinct runner ID and private URL; Node routes to that selected URL. The Runtime owns instance startup, ready capacity, withdrawal and cleanup (Deploy ordinary HTTP app images in gVisor and register them with Node #92/Verify sandboxed HTTP app execution through Node and the Python SDK #93). This does not require a session-specific Node routing API; Node #25 remains the separate multi-capability registration launch dependency.
- The orchestrator retains client-facing discovery, admission, proxy and payment validation. A separate redemption service is assumed to own transaction signing and the redemption key. Node needs keyless paid operation and durable winning-ticket handoff (livepeer/node #24); no recipient private key is mounted in allocator-managed pods. Clients supply signed tickets; there is no separate orchestrator ticket-signing requirement.
- Planned upgrades withdraw new work, preserve existing session traffic, allow a configurable drain window of up to 12 hours, then cancel remaining work and replace the pod (Implement delegated node management and orchestrator lifecycle with draining #94; livepeer/node #23). Current
Status != readyblocks existing proxies, so this is new Node work. An unplanned orchestrator restart loses process-local sessions/payment control state; stored winners must survive and resume handoff. - Platform qualification and allocator placement use ordinary deployment/drain controls. Node readiness/capacity describe runtime execution, not qualification policy (Derive qualification freshness and workload eligibility in the observability platform #91).
Still to specify
- Direct Kubernetes API versus an on-machine reconciler, exact delegated verbs and revocation (Define sandbox delegation, wallet-secret boundaries, and management topology #83/Prototype authenticated outbound management connectivity for NATed nodes #95); public job ingress direct versus tunnel (Evaluate sandbox public job ingress, DNS/TLS integration, and tunnel performance #96).
- Public
ServiceURLand privateRunnerServiceURL, DNS/TLS and route allowlists, callback paths, service/pod destination grants, and network policy enforcement on the selected CNI/gVisor stack (Enforce delegated sandbox permissions and protect operator secrets locally #85/Deploy ordinary HTTP app images in gVisor and register them with Node #92/Prototype authenticated outbound management connectivity for NATed nodes #95/Evaluate sandbox public job ingress, DNS/TLS integration, and tunnel performance #96). - Machine/account, deployment, pod instance and runner identity mapping; collision-free restarts/rollouts; which state is durable versus recreated (Add installer enrollment and the local machine credential lifecycle #87/Deploy ordinary HTTP app images in gVisor and register them with Node #92/Implement delegated node management and orchestrator lifecycle with draining #94).
- Create the new runtime/adapter repository and define its release contract, direct heartbeat lifecycle, health/capacity updates and 12-hour drain control (Deploy ordinary HTTP app images in gVisor and register them with Node #92; livepeer/node #23). Implement livepeer/node #25 before launch. Select compatible releases when implementing rather than fixing exact pins now.
- External redemption handoff contract, persistent data placement and recovery under orchestrator or redemption-service outage (livepeer/node #24).
Mapping completion (2026-10-30): The current protocol map, selected placement and credential split, and remaining design questions with owning tickets are recorded above. Concrete endpoints, identity and durable-state contracts, and tested recovery sequences are delivery work in #83/#85/#87/#92/#94/#95/#96 and livepeer/node #23/#24/#25. This source-mapping issue is complete; it does not claim deployed-cluster validation. Central allocator services and UI remain platform work.
Activity
j0sh commentedon Oct 30, 2026
Decision record from discussion (2026-10-29):
The issue body now distinguishes the pinned current behavior from this selected target. The remaining transport, endpoint, identity, adapter and redemption-handoff details are listed there and in their delivery tickets.
j0sh commentedon Oct 30, 2026
Closing this as a completed source-mapping and decision-handoff issue, consistent with #100 and #101. The current protocol/authority boundaries, selected orchestrator and app placement, direct adapter-to-Node heartbeat path, payment/redemption split, and unresolved design questions are recorded in the issue body.
The remaining work has delivery owners: delegation and credential boundaries (#83/#85), machine and runner identity (#87/#92), app deployment and adapter lifecycle (#92), drain and upgrade recovery (#94 and livepeer/node #23), management/public endpoints (#95/#96), external redemption handoff (livepeer/node #24), and multi-capability registration before launch (livepeer/node #25). Those tickets retain the design and test gates. Closing #99 does not assert that the sandbox has been deployed or validated.