🌐 Expose Renderflow through a first-class HTTP API and job service

[szmyty]

Overview

Expose Renderflow's reusable core engine through a first-class HTTP service so other Ego Hygiene products, local tools, automation, and future web applications can invoke the same transformation capabilities without shelling out to the CLI.

This should be an adapter over the stable Renderflow library API, not a second implementation of planning or execution logic.

Conceptually:

renderflow-core
    ├── renderflow-cli
    ├── renderflow-api
    └── downstream Rust tools

The CLI and HTTP service should consume the same request/result models and execution engine.

Motivation

Renderflow is becoming shared artifact infrastructure for publishing, animation, documentation, knowledge extraction, and other creative pipelines.

A network adapter makes those capabilities reusable by:

• web applications
• local services
• workflow orchestrators
• non-Rust consumers
• other Ego Hygiene products
• automation that needs asynchronous execution

The API must preserve the same deterministic planning, provenance, diagnostics, caching, validation, and safety semantics as direct library use.

Dependencies

Prefer implementing this after the core SDK/workspace boundaries from the Renderflow library refactor are stable.

Related work should be reviewed before implementation, especially:

• reusable Rust SDK/library
• modular workspace/core separation
• execution-plan and diagnostics APIs
• artifact capability/format model
• plugin/executor architecture

Scope

Design an optional renderflow-api crate or equivalent adapter.

Evaluate endpoints such as:

POST /v1/inspect
POST /v1/plan
POST /v1/execute
POST /v1/expand

GET /v1/jobs/{id}
DELETE /v1/jobs/{id}
GET /v1/jobs/{id}/artifacts

GET /v1/formats
GET /v1/transforms
GET /v1/capabilities

GET /health
GET /ready

The exact interface should follow the stable core API rather than forcing the engine to fit an arbitrary REST design.

Requirements

Shared engine behavior

The API must delegate to the Renderflow core library for:

• artifact inspection
• capability discovery
• graph planning
• optimization
• execution
• caching
• provenance
• output manifests
• diagnostics
• validation

No domain logic should be reimplemented in route handlers.

Structured contracts

Use explicit serializable request/result types.

Do not require API clients to parse terminal output.

Return structured:

• plans
• artifact manifests
• diagnostics
• warnings
• partial-success states
• validation results
• provenance references

Job execution

Long-running transforms should support asynchronous jobs.

Provide:

• job IDs
• status
• progress
• cancellation
• final result
• artifact references
• failure details

Progress streaming

Design for at least one streaming mechanism where useful, such as:

• Server-Sent Events
• WebSockets

Reuse adapter-neutral progress events from the core library.

Safety

Address:

• upload limits
• artifact size limits
• archive expansion limits
• request timeouts
• execution timeouts
• concurrency limits
• temporary workspace isolation
• path traversal
• overwrite behavior
• cancellation cleanup
• unsafe external process execution

Authentication

Keep authentication extensible.

A local-only deployment may not require authentication, but the API architecture must not prevent future bearer-token, reverse-proxy, or service-to-service authentication.

Do not hard-code a cloud identity provider.

Artifact delivery

Avoid loading arbitrarily large output artifacts into JSON responses.

Provide controlled artifact download/streaming semantics and clear lifecycle/retention behavior.

OpenAPI

Generate or maintain an OpenAPI description from the actual request/result contracts where practical.

The API schema should remain synchronized with implementation.

Observability

Integrate structured tracing and request correlation without exposing secrets or private artifact contents.

Deployment

Provide a minimal local/container deployment path.

Do not make a hosted Renderflow service a requirement.

Testing

Add coverage for:

• inspect
• plan
• synchronous small execution where supported
• asynchronous jobs
• cancellation
• partial success
• structured errors
• invalid uploads
• size/resource limits
• artifact retrieval
• missing external tools
• deterministic planning parity with the library and CLI

Documentation

Document:

• local startup
• API examples
• OpenAPI
• asynchronous job lifecycle
• artifact handling
• security assumptions
• resource limits
• embedding versus CLI versus API usage

Non-goals

This issue does not require:

• a public hosted Renderflow SaaS
• user accounts
• billing
• a web UI
• duplicating the CLI
• moving core behavior into the service layer

Acceptance Criteria

• HTTP/API adapter consumes the stable Renderflow core library.
• No planning or transformation logic is duplicated in route handlers.
• Artifact inspection and planning are available through structured endpoints.
• Long-running execution supports job status and cancellation.
• Progress can be consumed without terminal-specific behavior.
• Artifact retrieval is bounded and safe.
• Structured errors and partial-success states are exposed.
• Resource and upload limits are enforced.
• OpenAPI documentation is available and validated.
• Health/readiness endpoints exist.
• Local/container usage is documented.
• API tests pass alongside library and CLI tests.
• CI passes.