Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
4d28f95
includes all the files when ventis build
Saaketh0 Aug 26, 2026
23e3928
fixed some bugs
Saaketh0 Aug 26, 2026
95240ca
ventis build: sweep project .py files into Docker build contexts
Saaketh0 Aug 26, 2026
69d5405
Fix missing os import in metrics_agent.py
Saaketh0 Aug 26, 2026
4af43ae
[Feature] Pass env / secrets into agent containers
nickhuo Aug 26, 2026
087ee15
Harden the remote env file copy against a hostile /tmp
nickhuo Aug 26, 2026
905ef93
skill: porting an agent project onto Ventis
nickhuo Aug 26, 2026
a5e8f44
examples: joke_writer, a LangGraph map-reduce ported to Ventis
nickhuo Aug 26, 2026
c6e5516
Merge remote-tracking branch 'origin/feature/all-the-files' into jiaj…
nickhuo Aug 26, 2026
01a70f2
Fix PR #51 regression: stubs must also land flat
nickhuo Aug 27, 2026
bb7624e
Merge remote-tracking branch 'origin/feature/all-the-files' into jiaj…
nickhuo Aug 27, 2026
5d39ab2
Support managed-deployment secrets alongside self-hosted env_file
nickhuo Aug 27, 2026
d1ce61d
skill: make every MUST checkable, and add validate.py to check them
nickhuo Aug 27, 2026
b7e2f2f
examples/joke_writer: correct the policy comment, gate env_file on it…
nickhuo Aug 27, 2026
b33e4f0
Merge porting-to-ventis skill (PR #55, includes all-the-files) into c…
nickhuo Aug 27, 2026
c0e5ded
Merge env/secrets support (PR #53) into can-238 test harness branch
nickhuo Aug 27, 2026
c34e2a4
docs: design for testing porting-to-ventis across 100 repositories
nickhuo Aug 27, 2026
be72dba
docs: store artifacts rather than analysis; let validate not gate the…
nickhuo Aug 27, 2026
5f77b10
skill_harness: the eight-stage pipeline
nickhuo Aug 27, 2026
c1531bc
skill_harness: gate on the surfaces the credential can actually reach
nickhuo Aug 28, 2026
459f702
skill_harness: give subprocesses an interpreter, and stop scoring the…
nickhuo Aug 28, 2026
f89c12f
skill_harness: make the layout check mean what M24 means, and add a s…
nickhuo Aug 28, 2026
f9dd2d9
ventis build: install the project itself when it declares packaging m…
nickhuo Aug 28, 2026
80eb546
validate.py: probe the function stub_generator actually defines
nickhuo Aug 28, 2026
e4c98b0
skill_harness: the four repos that screen in scope, and why the other…
nickhuo Aug 28, 2026
c85bff6
docs: record what the corpus cost, and why can-228 was ported rather …
nickhuo Aug 28, 2026
db97b60
skill_harness: read the provider a repo picks at runtime, not just it…
nickhuo Aug 28, 2026
d4e3d01
skill_harness: do not outlive the harness
nickhuo Aug 28, 2026
d78a5b9
Revert "validate.py: probe the function stub_generator actually defines"
nickhuo Aug 28, 2026
3d3f1ac
ventis build: sweep the whole project, not only its modules
nickhuo Aug 28, 2026
7dd9289
skill_harness: talk to the providers directly, and stop rewriting any…
nickhuo Aug 28, 2026
8f660ff
skill_harness: read both provider signals, and screen out backing ser…
nickhuo Aug 28, 2026
fa51554
skill_harness: an openai-only repo that needs no backing service
nickhuo Aug 28, 2026
f112eca
skill_harness: record the tokens, and keep blocked rows out of the co…
nickhuo Aug 28, 2026
24160f6
skill_harness: separate the port serving from the project working
nickhuo Aug 28, 2026
55beca9
Replace the Python harness with a skill
nickhuo Aug 28, 2026
78645e1
testing-porting-to-ventis: ask what must be running, not what matches…
nickhuo Aug 28, 2026
579eae0
testing-porting-to-ventis: tear down, and check the ports before depl…
nickhuo Aug 28, 2026
9b7bf52
testing-porting-to-ventis: the port goes to a subagent, and its repor…
nickhuo Aug 28, 2026
ced63fd
ventis build: take PR #51's sweep and stub placement, and let -e . ca…
nickhuo Aug 28, 2026
91e23c4
porting-to-ventis: name the two imports a workflow needs
nickhuo Aug 28, 2026
9f4dfa0
porting-to-ventis: check the stub import, and stop W006 lying about -e .
nickhuo Aug 28, 2026
5922524
iterate the skill
nickhuo Aug 28, 2026
5639977
Rename porting skill to CanyonOS Core
nickhuo Aug 28, 2026
b826fa1
Remove testing porting skill from branch
nickhuo Aug 28, 2026
7ae76ab
Refine CanyonOS Core porting skill
nickhuo Aug 29, 2026
7620b01
Remove Ventis runtime changes from porting skill PR
nickhuo Aug 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 240 additions & 0 deletions .claude/skills/porting-to-canyonos-core/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
---
name: porting-to-canyonos-core
description: Ports existing LangChain, LangGraph, CrewAI, AutoGen, and hand-rolled Python agent projects to CanyonOS Core. Use when converting, migrating, adapting, packaging, building, or deploying an existing agent or multi-agent project onto CanyonOS Core.
compatibility: Requires Python, Docker, and the `ventis` compatibility CLI. Runtime identifiers remain `ventis`, `VENTIS_*`, and `ventis-*`.
---

# Port an agent project to CanyonOS Core

CanyonOS Core is the product name. Its compatibility executable and Python
package remain `ventis`; environment variables and Docker resources retain the
`VENTIS_*` and `ventis-*` prefixes. These are protocol identifiers, not branding
strings. Do not rename them.

## Load references only when needed

- Read [references/packaging.md](references/packaging.md) when a source import
does not resolve from `/app`, the source is nested, or packaging metadata is
involved.
- Read [references/llm-proxy.md](references/llm-proxy.md) only when the target
includes `llm_proxy`.
- Read [references/ec2.md](references/ec2.md) only when any config entry uses
`provider: EC2`.
- Read [references/troubleshooting.md](references/troubleshooting.md) after a
failed build, image probe, deploy, or request.
- Read [references/runtime-contract.md](references/runtime-contract.md) when a
validator finding needs explanation or the runtime mechanism is unclear.

## Goal: thin scaffolding beside untouched source

```text
agents/<name>.yaml one callable surface per service
agents/<name>.py one thin adapter per service, when needed
workflow/<name>_workflow.py HTTP entry point; calls deploy()
config/global_controller.yaml deployment manifest
config/policy.yaml optional access restriction
pyproject.toml conditional nested-import scaffolding
<source tree> unchanged
```

The file count follows the deployment. A multi-agent port has one yaml/adapter
pair per service that is worth deploying separately. If a source class already
satisfies the runtime contract, point its config entry at that file and do not
copy it into an adapter.

Everything the source already owns—prompts, tools, schemas, parsing, retries,
model clients, and node bodies—is imported. The port re-expresses only the
CanyonOS Core boundary and framework-owned orchestration.

The port root is the existing repository root and the directory from which
`ventis build` runs. Write scaffolding there beside existing directories. If the
repository already uses `src/`, leave it in place and put `agents/`, `workflow/`,
and `config/` beside it. Never move or copy the repository into a new `src/`
directory, and never create an outer wrapper merely for the port.

## 1. Survey before writing

Identify:

1. The source entry point and callable input/output.
2. Framework-owned control flow (`StateGraph`, `Crew`, `GroupChat`, routing,
`Send`, `Command`, interrupts).
3. Runtime-injected services nodes read: stores, context, memory, sessions, or
callback managers.
4. Sync versus async boundaries.
5. Imports and declared runtime dependencies.
6. Model provider, credential names, streaming use, and optional `llm_proxy`.
7. Whether independent work fans out and benefits from separate replicas.
8. Whether source imports resolve from the project root that becomes `/app`.

Run the validator once now. Its header detects capabilities directly from the
importable runtime rather than external development metadata:

```bash
python <skill_dir>/validate.py .
```

If config or agent yaml is malformed, the validator defers to `ventis build`.
Capability-gated findings say which runtime behavior is available.

## 2. Choose service boundaries

Start with one service. Split only when it creates independent parallel work or
a distinct resource/replica profile.

- Keep a ReAct loop together; every turn needs shared message history.
- Hoist supervisor task lists and `Send`-style fan-out into the workflow.
- Do not create a one-replica service with no distinct resource profile merely
to mirror every source graph node.

Rewrite framework-owned edges as ordinary Python. Import the connected node
functions unchanged. Construct runtime-injected service objects from source
configuration; do not invent models, dimensions, stores, or defaults silently.
Report any choice the source does not specify.

## 3. Write declarations and adapters

### Agent yaml

Use one yaml per deployed service. Argument types are bare builtins only:
`str`, `int`, `float`, `bool`, `dict`, or `list`. Every declared argument is
required by the generated stub. `returns.type` is documentation; use `dict` or
`list` to signal that workflow callers must `json.loads` the returned string.

### Adapter

The entrypoint exposes a module-level class named exactly `agent.name`. It
constructs with no arguments and its declared methods are synchronous. Read
configuration from the environment in `__init__`. Bridge source coroutines
inside a synchronous method with `asyncio.run(...)`. Serialize framework objects
with their own JSON-safe serializer before returning.

Do not duplicate source prompts, tools, schemas, or model calls. Keep the source
provider and SDK.

### Workflow

Expose `main(query: str)` and call `deploy(main, port=...)` at module scope.
Import generated stubs by yaml basename and agent class name:

```python
from deploy import deploy
from agents.<yaml_basename> import <AgentName>
```

The deployment platform sends `{query: string}` to `/main`. Pack richer input
inside `query`; any additional workflow parameter has a default.

Dispatch every remote call before resolving any future:

```python
futures = [agent.work(item=item) for item in items]
results = [json.loads(future.value()) for future in futures]
```

Do not fuse dispatch and `.value()` in one comprehension; that silently
serializes fan-out. Do not add an `if __name__ == "__main__":` block: the
workflow is executed with `__name__ == "__main__"` in production.

### Config

For each service, keep these names aligned:

```text
config entry name == yaml agent.name == entrypoint class name
```

Use lowercase `provider: local`; `replicas` is an integer; `requirements` is a
list of distribution-name strings. Put `env_file` at config top level when the
runtime capability is available. Omit `policy.yaml` unless access must be
restricted; if present, give it a non-empty `rules` list.

## Hard rules

Capitalized **MUST** and **NEVER** are reserved for port-breaking or
source-integrity rules. The owner column states where each is decided.

| ID | Rule | Owner |
|---|---|---|
| M1 | Entrypoint MUST define a class named exactly `agent.name` | V006 |
| M2 | The class MUST construct with no arguments | V007 |
| M3 | yaml argument names MUST match Python parameter names | V008 |
| M4 | yaml argument types MUST be bare builtins | V010 |
| M5 | Declared adapter methods MUST be synchronous | V009 |
| M6 | Config names MUST match yaml agent names | build |
| M7 | Config names MUST not collide after lowercase normalization | build output |
| M8 | Local provider MUST be lowercase `local` | deploy preflight |
| M9 | `replicas` MUST be an integer | deploy preflight |
| M10 | `requirements` MUST be a list of strings | build |
| M11 | Workflow MUST expose `main(query)`; extra parameters MUST default | V016 |
| M12 | Workflow MUST NEVER contain a main guard | V017 |
| M13 | Fan-out MUST dispatch all calls before resolving any | V018 |
| M14 | Project modules MUST not take runtime or generated-stub flat names | V019/V020 |
| M15 | Workflow MUST import stubs from `agents.<basename>` | V023 |
| M16 | Policy MUST be absent or contain a non-empty `rules` list | deploy preflight |
| M17 | EC2 entries MUST satisfy the EC2 deployment contract | deploy preflight |
| M18 | NEVER copy source prompts, tools, schemas, or model calls | review |
| M19 | NEVER hardcode or bake a real credential into an image | W003 |
| M20 | NEVER edit or vendor the source tree | `git status` |
| M21 | NEVER swap the source LLM provider | review |
| M22 | NEVER silently move, drop, or reclassify source dependencies | review |
| M23 | Framework control flow MUST be rewritten; source node logic MUST be imported | review |
| M24 | A non-resolving source import MUST have usable root packaging metadata when editable install is supported | V031 |

## 4. Validate, build, and probe

Run static preflight, then let the build own build-time validation:

```bash
python <skill_dir>/validate.py .
ventis build -c config/global_controller.yaml
```

A green build never imports the adapter. Probe each agent image in this order:

```bash
# Runtime startup path
docker run --rm ventis-<agent-name-lowercased> \
python -c "import local_controller"

# Agent load path; include --env-file when configured
docker run --rm --env-file <env-file> ventis-<agent-name-lowercased> \
python -c "import importlib.util,sys; \
s=importlib.util.spec_from_file_location('m','<entrypoint-basename>.py'); \
m=importlib.util.module_from_spec(s);sys.modules['m']=m;s.loader.exec_module(m); \
m.<AgentName>();print('ok')"
```

Also probe the workflow image with `python -c "import local_controller"`; it has
its own dependency resolve and generated-stub imports.

Then deploy, send a representative request, and poll its status:

```bash
ventis deploy -c config/global_controller.yaml
curl -X POST http://localhost:8080/main \
-H 'Content-Type: application/json' -d '{"query":"<real input>"}'
curl http://localhost:8080/status/<request_id>
```

A successful outer request with a source-level failure still proves the port
reached and returned the source behavior. Record the distinction.

## 5. Clean up

After collecting evidence, stop foreground deploy with Ctrl+C and wait for
controller cleanup. Remove exact leftovers if startup crashed. Then remove build
products and exact images from this config:

```bash
ventis clean
docker image rm ventis-<agent-name-lowercased> \
ventis-<workflow-entry-name-lowercased>

test ! -e stubs && test ! -e grpc_stubs && test ! -e docker_container
docker ps -a --format '{{.Names}}'
```

`ventis clean` removes only `stubs/`, `grpc_stubs/`, and `docker_container/`; it
does not remove containers or images. Keep port scaffolding, untouched source,
and requested logs or reports.
41 changes: 41 additions & 0 deletions .claude/skills/porting-to-canyonos-core/references/ec2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# EC2 deployment

Read this only when at least one config entry uses `provider: EC2`.

## Configuration

Use `provider: EC2` and declare `instance_type` on every EC2 service entry. The
top-level `ec2` block supplies the runtime's required infrastructure and SSH
settings. Read the target checkout's deploy preflight and EC2 runtime before
writing the block; do not copy values from an example environment.

Typical required categories are:

- AMI and instance type
- region and subnet
- security groups
- SSH user and credentials accepted by the runtime

`ventis deploy` owns basic EC2 config validation. A preflight pass is not proof
that provisioning, SSH, image transfer, or remote container startup works.

## Networking

A remote container's `host.docker.internal` names its own EC2 Docker host. It
does not name the local controller machine. Databases, model proxies, and other
services must use addresses reachable from every selected host.

The environment file may be copied temporarily to a remote host by runtimes that
expose the `env_file` capability. Confirm behavior from the capability probe and
target runtime rather than assuming local Docker semantics.

## Probes and cleanup

Run the same runtime and adapter probes against the exact image before remote
deployment. After deploy, verify the remote container logs; controller health
can be green even when agent loading failed.

Stop foreground deploy normally so the controller can terminate recorded EC2
instances. If provisioning or startup fails before an instance is recorded,
inspect the cloud provider directly and remove exact leaked resources. Never use
a broad cleanup command against unrelated instances.
64 changes: 64 additions & 0 deletions .claude/skills/porting-to-canyonos-core/references/llm-proxy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# LLM proxy integration

Read this only when the target checkout contains `llm_proxy` or the deployment
explicitly routes model SDKs through it.

## Preserve provider protocols

The proxy redirects provider endpoints; it does not convert providers. Keep the
source SDK, model ID, request body, and response parsing unchanged.

Configure only the provider variables the source uses:

```dotenv
OPENAI_BASE_URL=http://host.docker.internal:8081/openai/v1
ANTHROPIC_BASE_URL=http://host.docker.internal:8081/anthropic
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://host.docker.internal:8081/bedrock
```

Some SDKs refuse to initialize without caller credentials. Give agent containers
non-secret placeholders only when required. Keep real OpenAI, Anthropic, or AWS
credentials in the separate proxy process, not in the port's `env_file`.

## Start locally

The proxy defaults conflict with a typical deployment: host loopback is not
reachable from a container, and port 8080 is normally used by the workflow API.
Use a non-loopback bind and a different port:

```bash
PROXY_HOST=0.0.0.0 PROXY_PORT=8081 python -m llm_proxy
curl http://127.0.0.1:8081/healthz
```

Local CanyonOS Core containers resolve `host.docker.internal` through their
Docker host mapping. On EC2 that name resolves to each EC2 Docker host, not the
machine running `ventis deploy`. Distributed deployments need a reachable proxy
address or one proxy on each host.

## Supported call shape

The implementation buffers complete requests and responses:

- OpenAI and Anthropic non-streaming HTTP calls are forwarded.
- Bedrock `invoke` is reissued through the proxy's boto3 client.
- OpenAI/Anthropic streaming is unsupported.
- Bedrock `invoke-with-response-stream`, `converse`, and `converse-stream` are
unsupported.

Survey the source before selecting the proxy. Do not silently disable streaming;
report the unsupported behavior and stop.

## Credential behavior

- The OpenAI adapter removes caller authorization and inserts the proxy key.
- The Anthropic adapter removes caller key headers and inserts the proxy key.
- Botocore still signs requests sent to a custom endpoint, so a caller may need
placeholder AWS credentials even though the proxy reissues upstream with its
own identity.
- `/healthz` proves provider registration and Flask availability, not upstream
credential validity.

OpenAI and Anthropic upstream HTTP errors pass through. Proxy exceptions return
JSON 502 with `error: proxy_error`. Bedrock `ClientError` bodies are reconstructed
with the upstream status and are not byte-for-byte passthrough.
Loading
Loading