Anvil Agents

Product site: anvil-agents.hazyforge.io · Docs: anvil-agents.hazyforge.io/docs

Anvil Agents is an open-source Hazy Forge project for durable, distributed agent loops on Kubernetes. It turns independent, heavyweight agent work into schedulable Jobs, so repository builds, test suites, security analysis, indexing, migrations, and long research loops can use the CPU and memory of a cluster instead of competing on one workstation. Declarative runs, composable profiles, skill sets, tool sets, schedules, and event streams become isolated Jobs, and each run can select Codex, OpenCode, Hermes Agent, OpenClaw, Grok Build, Pi, or a custom harness.

Run/Profile/Harness/Skills/Schedule      Kubernetes Job
                 |                            |
                 v                            v
          anvil-agents controller -> selected harness -> tools and services
                 |                            |
                 +------- status/archive <----+
                 |
                 +------- optional OIDC read API and live SSE logs

Kubernetes provides scheduling, isolation, and distribution. Explicit AgentDataVolume resources and external services provide durable memory. Run profiles provide reusable role and policy. Harness, skill, and tool sets let operators change the runtime, instructions, and external integrations on independent lifecycles. The operator does not require Anvil Primaris, Anvil Hub, or another Hazy Forge control plane.

The branded API group control.anvil.hazyforge.io/v1alpha1 is retained as a stable API identity and for migration compatibility. Application and target references are opaque scope keys, not dependencies on other CRDs.

Test It Without Rebuilding

On Linux amd64 with Docker Engine running, install the pinned user-space tools and run the test:

./hack/install-judge-prerequisites.sh --install \
  --bin-dir "${HOME}/.local/bin"
export PATH="${HOME}/.local/bin:${PATH}"
./hack/install-judge-prerequisites.sh --check
./hack/test-judge-kind.sh

The prerequisite installer downloads the certified Kind, kubectl, and Helm versions from their official release hosts, verifies pinned SHA-256 checksums, and never uses sudo. It checks Docker but does not install or reconfigure the privileged host service. The judge script builds nothing and needs no API key. It creates a dedicated Kind cluster, installs the public v0.1.1 OCI chart with the controller pinned to its published digest, and proves two append-only AgentRun Jobs share durable PVC state while producing structured terminal status. Getting started covers check-only mode, custom installation directories, expected output, inspection, troubleshooting, and cleanup.

The validated release platform is Linux amd64, Kubernetes 1.30 or newer, and Helm 3.14 or newer. Arm64 images and older Kubernetes releases are not part of the certified submission contract.

What It Owns

• AgentRun: one append-only execution record and one harness Job.
• AgentRunProfile: reusable role, scope, policy, and composition defaults.
• AgentHarnessProfile: reusable backend and Kubernetes execution envelope.
• AgentSkillSet: reusable skills and delegated personas.
• AgentToolSet: reusable external tool setup and verification contracts.
• AgentCouncil: durable workforce inventory and opt-in interaction guidance; it does not merge member authority or create runs.
• AgentSchedule: interval and manual run creation across named templates.
• AgentExternalTrigger: signed GitHub webhooks that create or annotate runs (docs/external-triggers.md).
• AgentRunControl: cluster-wide pause and concurrency policy by scope key.
• AgentDataVolume and VolumeProfile: explicit durable PVC-backed state.
• AgentAuthSession: operator-driven durable Codex auth reauth/logout sessions.
• AdverseSituation: deduplicated event buffers with optional responders.
• AdverseSignal: immutable, provider-neutral adverse evidence from any app.
• anvil-agents-api: optional OIDC-protected summaries, live SSE logs, and the read-only Anvil Agents Console SPA at / (see web/console/).
• anvil-agentctl: Kubernetes-authenticated fleet CLI for runs (create/list/ get/logs/debug), AgentRunControl launch gates (control pause|resume), schedules (schedule list|suspend|resume|run-now), durable-home auth, volume copy, and in-pod status reporting. Private Primaris anvilctl has no agent subcommand; managers use the Hub HTTP API.
• anvil-desktop: Anvil Agents Desktop, a loopback process that signs in to the anvil-agents OIDC API (Anvil Primaris) and talks to cluster agents. Chat and Wrapper are the product; Local harness activation is a second page (see web/desktop/ and Anvil Agents Desktop).
• Collector-neutral Job and Pod labels for external log and telemetry pipelines.

"Multi-harness" means every run uses one adapter behind a common payload, tool-bootstrap, and status contract. Schedules can distribute independent runs across harnesses and nodes. It does not mean the controller creates an agent mesh or a shared live conversation; cross-run state must be explicit.

Use The Whole Cluster

Heavy agent work is often more than model inference. Agents compile large repositories, run integration suites, scan dependency graphs, build images, query local indexes, and process large evidence sets. Anvil Agents lets an operator turn those independent workloads into a controlled cluster queue:

• Harness profiles declare CPU, memory, and ephemeral-storage requests and limits instead of inheriting the capacity of the machine that submitted the run.
• Node selectors, affinity, and tolerations place build, memory-heavy, custom accelerator, or storage-local harnesses on the machines prepared for them; accelerators also require a device plugin and extended-resource request.
• Multiple AgentRun objects and AgentSchedule with Allow can execute parallel lanes, bounded by schedule and application concurrency controls.
• Kubernetes performs placement and isolation, while run status and the OIDC stream let users observe work without SSH or kubectl access.

One AgentRun is one Pod on one node; the operator scales heavy work horizontally by scheduling many independent runs across machines. A harness that needs to split one computation across several nodes must use an explicit distributed compute service. See Distributed Workloads for placement, concurrency, storage, and capacity examples.

Try It Locally

The credential-free Kind quickstart builds the controller and a minimal custom harness, installs the chart, binds and mounts durable state, creates an AgentRun, and waits for structured success:

./examples/quickstart/run.sh

See Getting Started for prerequisites, manual steps, existing-cluster installation, and cleanup.

Operators with Kubernetes credentials use anvil-agentctl as the runtime CLI: create one append-only run from a profile, pause/resume application launch gates, suspend or nudge schedules, inspect status, stream the verified runner container, and aggregate controller, Job, Pod, and Event evidence. See AgentRun CLI. Product managers on Anvil Primaris call the Hub agent-management HTTP API for brokered mutations (no anvilctl agent).

Build Without GitHub Actions

Local scripts and Make targets are the canonical build and validation entry points:

make verify
make images
make kind-e2e

./hack/build-images.sh --component controller --tag test
./hack/build-images.sh \
  --prefix registry.example.com/team \
  --tag v0.1.1 \
  --push

./hack/package-chart.sh --version 0.1.1

# Complete local publication from a clean checkout after docker login and
# helm registry login:
make release-local-all VERSION=vX.Y.Z REGISTRY_PREFIX=registry.example.com/team

# Update the first-party Anvil Primaris deploy overlay from the generated lock:
make release-pin-deploy VERSION=vX.Y.Z

make images builds the controller plus all eight built-in runner images into local Docker. The reusable script supports component selection, platforms, cache import/export, multiple tags, custom registries, and fork-aware OCI source metadata. Image pushes reject dirty worktrees unless explicitly overridden. make release-local-all creates or verifies an annotated release tag, pushes that tag, runs publish-release.sh, then creates the GitHub Release page from the local chart package and image lock. The GitHub Release step uses the GitHub API through gh, verifies that the remote tag already exists, and does not use GitHub Actions minutes. If you only need the registry and chart artifacts, run make release-local.

publish-release.sh runs make verify and make kind-e2e, publishes all nine versioned images, verifies their immutable digests and source revision, writes a digest lock, and pushes an OCI chart whose nine default image references are pinned to that lock. make release-pin-deploy updates the first-party Anvil Primaris overlay from that lock so the controller and built-in runner defaults move together.

GitHub workflows use the same repository-owned build and test contracts and are optional. The optional release workflow is manual-only for an existing v* tag. It verifies the tag and Kind upgrade/install tests, then invokes publish-release.sh --skip-verification so Actions and local publication produce the same digest-locked chart. It does not run for tag pushes or master, so the local GitHub Release step cannot accidentally start a second publisher for the same version.

Security Boundary

Run, run-profile, harness-profile, and skill-set authors can cause executable code to run. Harness profiles can also choose a same-namespace ServiceAccount, Secrets, resource placement, storage, and a custom command. Treat permission to edit these resources as privileged workload execution. Use dedicated namespaces, narrowly scoped runner identities, admission policy, and Secrets created only for agents. watchNamespaces narrows the controller cache; it does not narrow the chart's ClusterRole.

The OIDC API is a separate read-only workload and is disabled until an issuer, audience, and explicit namespace authorization bindings are configured. See Security and Live AgentRun API.

Documentation

• Blog: Agents as cluster jobs, not chatbots
• Product site and docs — public marketing site and operator docs
• Architecture and multi-harness semantics
• External triggers (GitHub webhooks)
• Distributed heavy workloads across machines
• Composable profiles, harnesses, skill sets, tool sets, councils, and overrides
• Getting started
• Local simulator with pre-seeded agents
• Harness contract and adapter matrix
• Knowledge bases, tools, and external services
• AgentRun stdout, Alloy, and OpenTelemetry integration
• Connect any application or Kubernetes API to adverse streams
• Operations, upgrades, and uninstall
• PostgreSQL archive modes and retention
• Standing chat storage and API
• One chat across local and cloud sessions
• Design roadmap and known alpha boundaries
• AgentRun API reference
• Chat mailbox contract (interactive purpose)
• Create and diagnose runs with anvil-agentctl
• Anvil Agents Desktop
• Install Anvil Agents Desktop
• Migration from Anvil Primaris
• Third-party notices

Hazy Forge uses this same open-source runtime for its own agent system. The repository-local .hazyforge/artifact-build.yaml and .hazyforge/tests.yaml files are maintainer build and test contracts. The .hazyforge/clusters/anvil-primaris/ tree is an optional Hazy Forge consumer deployment with environment-specific identity, credentials, routing, storage, placement, and image pins. None of those files is a chart default or runtime dependency; other consumers provide their own Helm values or GitOps layer. This repository does not advertise cluster agent roles or ship a self-development fleet under .hazyforge/agents/.

Project Status

The API is v1alpha1. Pin chart and image versions, test upgrades against a cluster backup, and expect compatible additions before a stable API release. The initial v0.1 release contract targets Linux amd64. Source builds may work on arm64, but arm64 is not yet part of the published release gate.

Licensed under Apache-2.0. See Contributing, Security Policy, and Support.