Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/triage-backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ cd - && git worktree remove --force <tmp>/port

`upstream-merge.py` has no dry-run mode and writes files, so this second throwaway worktree is the dry run. Report its verbatim, clean-merge and needs-review counts. Exit 1 means review is needed. Exit 2 means the script refused its input. Report the two apart.

Tag each item port-only or Cursor-derived, by the paths it edits (a PR's files) or names, as `UPSTREAM.md` and `NOTICE.md` map them; an item that names none is tagged from its subject, and the tag says so. Before a local fix to a Cursor-derived file is scheduled, check the source that `UPSTREAM.md` or `NOTICE.md` names for that path for an existing fix, such as `git log cursor/main -- pstack/<path>` for the pstack tree. An "upstream first" disposition is finished only when the authorized upstream action exists and its link is in the plan. For an issue already closed upstream-first, record whether the upstream PR exists, and act only when its reopen condition fired. Route a new tool as an external provider, not as a parent harness. Done when the counts, the exit code and a tag for each item are in the plan.
Tag each item port-only or Cursor-derived, by the paths it edits (a PR's files) or names, as `UPSTREAM.md` and `NOTICE.md` map them; an item that names none is tagged from its subject, and the tag says so. Before a local fix to a Cursor-derived file is scheduled, check the source that `UPSTREAM.md` or `NOTICE.md` names for that path for an existing fix, such as `git log cursor/main -- pstack/<path>` for the pstack tree. An "upstream first" disposition is finished only when the authorized upstream action exists and its link is in the plan. For an issue already closed upstream-first, record whether the upstream PR exists, and act only when its reopen condition fired. Distinguish model providers from parent harnesses; T3 Code is a supported parent harness. Done when the counts, the exit code and a tag for each item are in the plan.

### Steps 2 (Collect) and 5 (Release lag): model snapshot, release procedure and model lag

Expand Down
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"name": "pstack",
"source": "./plugins/pstack",
"description": "if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence.",
"version": "1.18.0",
"version": "1.19.0",
"author": {
"name": "Lauren Tan (original)"
},
Expand Down
4 changes: 3 additions & 1 deletion .claude/skills/verify-open-pstack/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: verify-open-pstack
description: Verify an Open Pstack PR's exact candidate head in native Claude Code and Codex sessions, judge them with machine-checked assertions, and publish the exact-SHA live gate.
description: Verify an Open Pstack PR's exact candidate head in native Claude Code, Codex and T3 Code sessions, judge them with machine-checked assertions, and publish the exact-SHA live gate.
---

# Verify Open Pstack
Expand Down Expand Up @@ -51,6 +51,8 @@ Create detached exact-head candidate checkouts and run-owned temporary/fixture s

Claude runs with normal operator config/login and session-only `claude --plugin-dir <exact-head checkout>/plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'`, plus `-p --input-format stream-json --replay-user-messages --output-format stream-json --verbose --no-session-persistence --permission-mode dontAsk --allowedTools <recipe tools>`. The prompt arrives on stdin, so the replayed user message shows the slash expansion. A tool outside the pre-authorized list is denied without a prompt. Codex runs `codex exec --json` against the run-owned installation with approvals set to never, `--sandbox workspace-write`, `--add-dir` for a fixture outside its working directory, and `--dangerously-bypass-hook-trust` so the candidate's hooks run for that invocation only. Cases whose parent launches an external provider CLI use `--sandbox danger-full-access`, because those CLIs need network and their own state outside the fixture. Codex installs the exact-head marketplace/plugin in its run-owned home with `codex plugin marketplace add … --ref <sha>` and `codex plugin add`. Verify candidate plugin and project-skill sources against immutable Git provenance before and after exercise; version strings alone are insufficient.

`t3:` features run through the T3 transport in `features/t3.md`: a run-owned disposable T3 server whose Claude provider gets the same session-only plugin flags, one `t3 thread start --wait --json` per case, and the parent's Claude transcript parsed like the Claude stream. Classification never adds `t3` on its own and self-test does not reach it; the doctor checks the T3 service binary only when a `t3:` feature is named.

Candidate commands and provider descendants use the recorded native-login arrangement, never GitHub publisher credentials. No extra login, implicit timeout, or weaker-model fallback is permitted. If Claude's plugin flags fail to exclude installed pstack or Codex replaces the auth symlink instead of writing through it, stop and reopen the design.

For every named feature, exercise its native surface and retain real native tool calls and concrete fixture effects. Asset consumers come from the pinned plugin manifests; the current `plugins/pstack/assets/logo.png` is consumed by the Codex manifest only and therefore requires the Codex installed surface, not an invented Claude asset exercise. The session recorder streams each session's stdout and stderr to private files under `sessions/<harness>/<feature>/` in the run root and keeps them on any exit. For Codex it also copies the session's rollout from the run-owned home, because only the rollout records which `SKILL.md` each skill mention resolved to. A nonzero session exit fails the case. The trusted parent then parses the native stream and evaluates the case's assertions on the native skill load, the commands that ran, and files the session wrote in its fixture, and copies those files next to the stream. A model's claim that something passed never counts, and neither do direct CLI tests.
Expand Down
1 change: 1 addition & 0 deletions .claude/skills/verify-open-pstack/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ The feature documents below define sub-features, user entry points, driving inst
- `catalog.md`: the generated model catalog and native agents, read by a parent resolving roles from a model sheet.
- `shipped-tools.md`: watch, plan checks, audit, statechart, and evidence logging.
- `assets.md`: manifest-declared installed assets on actual consuming harnesses.
- `t3.md`: a T3 Code parent dispatching a role through `delegate_task` (`t3:dispatch`), on a run-owned disposable T3 server.
- `hooks.md`: the SessionStart hook's context reaching a new session (`hook:startup`), and the Windows runner beside it.
- `recipes.ts`: the headless cases and machine-checked assertions for every selectable feature. A named feature without one fails the run as `missing-recipe` before any session starts. A recipe proves a lookup or a read with a fixture value the model cannot derive from the prompt, such as a per-run `crypto.randomUUID()` marker, compared in the structured result. A command's text can show which tool the parent called, but it never proves what that tool read or produced, so do not add command-text or tool-output pattern checks for that.
- `agent-preload:poteto-agent` (Claude only): the parent dispatches `pstack:poteto-agent` through the Agent tool and asks it for the first Core principle of the preloaded poteto-mode skill without invoking `Skill` or reading files. The check needs that Agent call, `Laziness Protocol` in the reply the parent writes to `result.md`, and no `Skill` call or `Read` of poteto-mode's `SKILL.md` among the child's own tool calls.
Expand Down
38 changes: 34 additions & 4 deletions .claude/skills/verify-open-pstack/features/recipes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { homedir } from 'node:os';
import { basename, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { sha256, type Command } from '../scripts/io.ts';
import type { Harness, SessionRecord } from '../scripts/types.ts';
import type { CoreHarness, Harness, SessionRecord } from '../scripts/types.ts';

export interface Fixture {
harness: Harness;
Expand All @@ -29,7 +29,7 @@ export interface Case {
}

export const CLAUDE_TOOLS = 'Skill,Read,Glob,Grep,Write,Edit,Bash,Agent,Task,TodoWrite';
export const DEFAULT_ROLES: Record<Harness, string> = { claude: 'bug-fix', codex: 'judgment and prose' };
export const DEFAULT_ROLES: Record<CoreHarness, string> = { claude: 'bug-fix', codex: 'judgment and prose' };
export const ROUTE = /^([a-z][a-z0-9-]*):([A-Za-z0-9._/-]+)@([a-z]+)$/;
const SETUP_DESCRIPTOR = 'claude:opus@high';
const DESCRIPTOR = /\b[a-z][a-z0-9-]*:[A-Za-z0-9._/-]+@[a-z]+/g;
Expand All @@ -54,7 +54,7 @@ export async function readCatalog(location: string): Promise<Catalog> {
const kept = new Map(rows(section('Kept predecessors and normalization').split('\nRolling aliases:')[0]!).slice(1)
.map(([provider, model, , current]): [string, string] => [`${provider}:${model}`, `${provider}:${current}`]));
const [header = [], ...routes] = rows(section('Routes'));
const column = { claude: header.indexOf('Claude Code parent'), codex: header.indexOf('Codex parent') };
const column = { claude: header.indexOf('Claude Code parent'), codex: header.indexOf('Codex parent'), t3: -1 };
if (!defaults.size || column.claude < 0 || column.codex < 0) throw new Error('catalog-unreadable');
return { defaults, kept, route(harness, descriptor) {
if (!descriptor.includes(':')) return 'native';
Expand Down Expand Up @@ -440,7 +440,7 @@ const museReader: Case = {
},
};

function runner(harness: Harness, routes: string[]): Case[] {
function runner(harness: CoreHarness, routes: string[]): Case[] {
const chosen = routes.filter(route => !route.startsWith(`${harness}:`));
const cases = (chosen.length ? chosen : ['']).map((fixed, index): Case => {
let route = fixed, provider = '', model = '', effort = '';
Expand Down Expand Up @@ -775,7 +775,37 @@ const startupHook: Case = {
},
};

const T3_ROLE = 'bug-fix', T3_LANE = { provider: 'claudeAgent', model: 'claude-haiku-5-5', effort: 'low' };
const t3Dispatch: Case = {
id: 'dispatch',
async prepare(f) {
await files(f.dir, { 'marker.txt': `${crypto.randomUUID()}\n`, 'pstack-models.md': sheetText([[T3_ROLE, [`claude:${T3_LANE.model}@${T3_LANE.effort}`]]]) });
},
prompt: f => `${invoke(f.harness, 'poteto-mode', `Dispatch exactly one lane for the ${T3_ROLE} role and nothing else. For this task the pstack model sheet is ${join(f.dir, 'pstack-models.md')}; use it in place of any configured sheet. The lane's whole task: "Read ${join(f.dir, 'marker.txt')} and reply with its exact contents. Do not edit files." Wait for the lane's result, then reply with it.`)} ${rules(f.dir)}`,
async assert(record, f) {
if (!record.skills.includes(join(f.location, 'skills/poteto-mode'))) throw new Error('skill-not-loaded-from-candidate:pstack:poteto-mode');
const calls = record.tools.filter(t => t.name === 'mcp__t3-code__delegate_task' && !t.failed);
if (calls.length !== 1) throw new Error(`delegate-task-calls:${calls.length}`);
const { input, result } = calls[0]!, target = input.target ?? {};
if (target.providerInstanceId !== T3_LANE.provider || target.model !== T3_LANE.model || !Bun.deepEquals(target.options, { effort: T3_LANE.effort })) throw new Error('delegate-request-mismatch');
const response = parseJson(result) as Record<string, any> | undefined;
if (response?.providerInstanceId !== T3_LANE.provider || response.model !== T3_LANE.model ||
!Bun.deepEquals(response.options, [{ id: 'effort', value: T3_LANE.effort }])) throw new Error('delegate-response-mismatch');
if (response.status !== 'completed' || response.workState !== 'result_available' || response.hasPendingChildRuns !== false ||
response.hasPendingQuestion !== false || response.pendingApproval !== null) throw new Error(`delegate-not-settled:${response.status}`);
Comment thread
ericlitman marked this conversation as resolved.
Resolved
Comment thread
github-actions[bot] marked this conversation as resolved.
Resolved
const marker = (await readFile(join(f.dir, 'marker.txt'), 'utf8')).trim();
if (typeof response.summary !== 'string' || !response.summary.includes(marker)) throw new Error('marker-missing');
const exchange = join(f.dir, 'delegate-task.json');
await writeFile(exchange, JSON.stringify({ request: input, response }, null, 2));
return { assertions: ['skill-loaded-from-candidate', 'delegate-request', 'delegate-settled', 'marker-returned'], files: [exchange, join(f.dir, 'pstack-models.md')] };
},
};

export function recipe(feature: string, harness: Harness, routes: string[] = []): Case[] {
if (harness === 't3') {
if (feature === 'dispatch') return [t3Dispatch];
throw new Error(`missing-recipe:${harness}/${feature}`);
}
if (feature === 'assets:codex') throw new Error('unsupported-native-consumer:assets:codex');
if (feature === 'project-skill') return [projectSkill];
if (feature === 'hook:startup') return [startupHook];
Expand Down
21 changes: 21 additions & 0 deletions .claude/skills/verify-open-pstack/features/t3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# T3 parent dispatch

## Sub-features

`t3:dispatch`: a T3 Code parent running the installed `poteto-mode` routes a role whose provider T3 offers through T3's `delegate_task`, with the model and effort from the model sheet, and judges the result by the T3 host completion rules in `provider-dispatch.md`. Select it for a change to `provider-dispatch.md`, `codex-tools.md`, `setup-pstack/SKILL.md` or `recipes.ts`. A path that reaches it never adds it to every run; name it with `--feature t3:dispatch`.

## How to get to it (user POV)

Start a T3 Code thread on the Claude provider with the candidate plugin loaded, and ask `poteto-mode` to dispatch one role from a model sheet.

## Driving it with verify-open-pstack

The `t3` transport runs a disposable T3 server from the operator's installed service binary, `~/.t3/runtime/versions/<activeVersion>/t3`, which `~/.t3/runtime/service-state.json` names. The server's base directory and project live in the run root. Its `userdata/settings.json` sets the `claudeAgent` provider's `launchArgs` to the same session-only `--plugin-dir <exact-head checkout>/plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'` the Claude harness uses, so the parent and every Claude child it delegates to load the candidate. Each case adds the fixture as a project, runs its prompt with `t3 thread start --provider claudeAgent --model claude-sonnet-5-5 --wait --json`, and stops the server.

## Gotchas

`t3 thread show --json` carries neither tool calls nor options, so the transport reads the parent's Claude transcript. The run-owned `statev2.sqlite` maps the T3 thread to its Claude session id (`nativeThreadRef.nativeId`), and the transcript is `<Claude config>/projects/<project realpath, each non-alphanumeric character as ->/<session>.jsonl`. That transcript and the delegated child's stay in the operator's Claude projects folder; the verifier copies the parent's into the run root. The operator's T3 service, settings and data are never touched.

## Headless

The `dispatch` case writes a fixture sheet whose `bug-fix` role is `claude:claude-haiku-5-5@low` and a `marker.txt` holding a per-run `crypto.randomUUID()` that the prompt never contains. It asserts the transcript's `Base directory for this skill` line for `poteto-mode` is the candidate plugin path; exactly one `delegate_task` call has `target.providerInstanceId` `claudeAgent`, `target.model` `claude-haiku-5-5` and `target.options` `{"effort":"low"}`; its result names the same provider, model and effort and is settled (`completed`, `result_available`, no pending child runs, question or approval); and its summary holds the marker. The request and response are retained as `delegate-task.json`. The model's prose is never read.
2 changes: 1 addition & 1 deletion .claude/skills/verify-open-pstack/scripts/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export function parse(args: string[]): Options {
if (!ROUTE.test(value) || routes.includes(value)) throw new Error('Runner route must be a distinct PROVIDER:MODEL@EFFORT');
routes.push(value);
} else if (arg === '--feature') {
if (!/^(claude|codex):[a-z][a-z0-9:-]+$/.test(value) || features.includes(value)) throw new Error('Feature must be a distinct HARNESS:FEATURE, such as claude:agent-preload:poteto-agent');
if (!/^(claude|codex|t3):[a-z][a-z0-9:-]+$/.test(value) || features.includes(value)) throw new Error('Feature must be a distinct HARNESS:FEATURE, such as claude:agent-preload:poteto-agent');
features.push(value);
} else if (arg === '--wide') {
if (!value.trim() || !publishable(value)) throw new Error('Wide reason must be nonblank printable ASCII, at most 160 characters, without backticks or angle brackets');
Expand Down
18 changes: 15 additions & 3 deletions .claude/skills/verify-open-pstack/scripts/core.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { resolve } from 'node:path';
import registryData from '../features/registry.json';
import { recipe } from '../features/recipes.ts';
import { classify, completeEvidence, matches, newReceipt, requiredFeatures, requiredHarnesses, validateRegistry } from './core.ts';
import { HARNESSES, type Receipt } from './types.ts';
import { HARNESSES, LIVE_HARNESSES, type Receipt } from './types.ts';
import codexManifest from '../../../../plugins/pstack/.codex-plugin/plugin.json';
import claudeManifest from '../../../../plugins/pstack/.claude-plugin/plugin.json';

Expand Down Expand Up @@ -63,6 +63,18 @@ describe('registry and ownership', () => {
expect(s.features).toContain('claude:agent-preload:poteto-agent');
expect(s.features).not.toContain('codex:agent-preload:poteto-agent');
});
test('only the T3 dispatch surfaces reach t3:dispatch, and self-test never adds a T3 session', () => {
const reach = (filename: string) => classify([{ filename }], registry).features.includes('t3:dispatch');
for (const filename of ['plugins/pstack/skills/poteto-mode/references/provider-dispatch.md', 'plugins/pstack/skills/poteto-mode/references/codex-tools.md',
'plugins/pstack/skills/setup-pstack/SKILL.md', '.claude/skills/verify-open-pstack/features/recipes.ts']) expect(reach(filename)).toBe(true);
for (const filename of ['plugins/pstack/skills/poteto-mode/references/model-catalog.md', 'plugins/pstack/skills/poteto-mode/SKILL.md',
'plugins/pstack/hooks/hooks.json', '.claude/skills/verify-open-pstack/scripts/harness.ts']) expect(reach(filename)).toBe(false);
const r = receipt(true);
expect(requiredHarnesses(r)).toEqual(['claude', 'codex']);
r.runFeatures = ['t3:dispatch'];
expect(requiredHarnesses(r)).toEqual(['claude', 'codex', 't3']);
expect(requiredFeatures(r, 't3')).toEqual(['dispatch']);
});
test('poteto-mode SKILL.md also reaches the Claude agent that preloads it', () => {
const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/SKILL.md' }], registry);
expect(s.features).toEqual([...both('skill-invocation:poteto-mode'), 'claude:agent-preload:poteto-agent'].sort());
Expand Down Expand Up @@ -187,13 +199,13 @@ describe('registry and ownership', () => {
test('the shared recipe implementation reaches every implemented recipe and preserves project proof', () => {
const path = '.claude/skills/verify-open-pstack/features/recipes.ts';
const expected = [...both('catalog', 'setup', 'runner', 'shipped-tools', 'hook:startup', 'project-skill', 'writers',
...registry.skills.map(skill => `skill-invocation:${skill}`)), 'claude:agent-preload:poteto-agent'].sort();
...registry.skills.map(skill => `skill-invocation:${skill}`)), 'claude:agent-preload:poteto-agent', 't3:dispatch'].sort();
for (const file of [{ filename: path }, { filename: 'docs/moved.md', previous_filename: path }]) {
const s = classify([file], registry);
expect(s.features).toEqual(expected);
expect(s.skills).toEqual([...registry.skills].sort());
expect(s.noRuntime).toBe(false);
for (const harness of HARNESSES) {
for (const harness of LIVE_HARNESSES) {
for (const feature of s.features.filter(f => f.startsWith(`${harness}:`))) {
expect(recipe(feature.slice(harness.length + 1), harness).length).toBeGreaterThan(0);
}
Expand Down
Loading
Loading