diff --git a/README.md b/README.md index 75a12f2..7fb8c4d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Claude Code mods (plugins with a hooks module), served as a local folder marketp | --- | --- | --- | | [goal-mode](goal-mode/) | 0.7.0 | Session goals, handed to Claude Code's own `/goal`: set by you (`/goal-mode set [--check ]`) or by Claude itself (nudged on outcome requests and approved plans); its own evaluator judges only the loop's queued goals | | [harness](harness/) | 0.3.2 | The integrated harness: the user's Claude rules enforced at the tool call, session tooling and workflow automation (formerly house-rules) | -| [auto-claude](auto-claude/) | 0.3.0 | Unattended run of a repository on top of goal-mode's loop: plans the long-term goal and milestones, ships each (push, PR, CI, merge when green, then deploy where `.claude/auto-deploy.json` opts in), replans until the final check passes | +| [auto-claude](auto-claude/) | 0.4.1 | Unattended run of a repository on top of goal-mode's loop: plans the long-term goal and milestones, ships each (push, PR, CI, merge when green, then deploy where `.claude/auto-deploy.json` opts in), replans until the final check passes | | [model-tier](model-tier/) | 0.11.0 | Routes each main-loop turn to fable, opus, sonnet or haiku and a reasoning effort by task complexity, with a `set_tier` tool the model calls to correct either | | [refactor](refactor/) | 0.1.3 | Enforces the architecture a repo declares in `.claude/architecture.json` at Edit/Write (layer direction, forbidden imports), notes code smells with the refactoring.guru technique to apply, `/arch audit` for violations and import cycles | diff --git a/auto-claude/.claude-plugin/plugin.json b/auto-claude/.claude-plugin/plugin.json index fcee784..0259e4a 100644 --- a/auto-claude/.claude-plugin/plugin.json +++ b/auto-claude/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "auto-claude", - "version": "0.3.0", + "version": "0.4.1", "description": "Finishes a repository's work unattended: Claude plans the long-term end state and its milestones, goal-mode's loop builds each one in a worktree, and auto-claude ships it (push, PR, CI, merge when green), then replans until the final check passes.", "author": { "name": "unohee" diff --git a/auto-claude/README.md b/auto-claude/README.md index ec1a86a..24ef85c 100644 --- a/auto-claude/README.md +++ b/auto-claude/README.md @@ -9,6 +9,12 @@ An unattended run of one repository. Give it an aim in one line, or none at all. It depends on [goal-mode](../goal-mode/), which supplies the loop: one worktree per milestone, checks, the evaluator, stuck handling and guards. +## Starting anywhere + +- **Not a git repository.** `/auto-claude start` in a plain directory runs `git init -b main`, adds secrets and noise to `.gitignore` (`.env*`, `*.pem`, `*.key`, `node_modules/`, `worktrees/`, `.claude/settings.local.json`), and makes one initial commit of what is there. Plan the empty case too: with no files, the first milestone is the scaffold. +- **Refused.** The home directory, `/`, a directory with more than 3000 files, and a directory that holds other repositories are not one project, so nothing is initialized there. +- **No `origin` remote** (a new repository, or an existing one). Milestones land in the repository's own default branch: the check, a rebase onto it, then `git push . HEAD:` with `receive.denyCurrentBranch=updateInstead`, which updates the checked-out files in place. There is no PR and no CI, and `.claude/goal-loop.base` is the branch itself. An existing repository must have a clean tracked tree to start. + ## A run 1. **Plan.** @@ -54,6 +60,8 @@ A repository opts in by committing `.claude/auto-deploy.json`: - A `/goal-mode loop on` that starts nothing is retried with backoff. After 3 tries the person is told the loop's reply. - A run still planning gets a reminder every 15 minutes that lists what the plan files lack. After 3 reminders the run stops. - A shipping error that repeats is reported once. + - A tick that fails 3 times in a row (an unwritable `.claude/goal-loop.gate`, say) is reported once with its error, and `stalled: every tick fails` stays in `/auto-claude status` until a tick gets through. + - **Idle nudge.** A run on the `run` phase where nothing was started, shipped or merged for 15 minutes is reported to the person with its cause (the `stalled:` reason, or "idle N min"), again 30 minutes later, and once more, 3 nudges in all (`idleNudges` in the record; any progress resets the count). When the loop is the idle one, the nudge also clears the `loop on` backoff so the grant is asked at once. A goal being built is waiting, not idle. So is a PR whose CI is pending, or whose mergeability GitHub has not decided, for up to 90 minutes: after that the wait is a stall (a check queued for an offline self-hosted runner never finishes), named `PR #N has waited M min for CI` in the nudge and in `stalled:`. The session model is not prompted. - `/auto-claude status` ends with a `stalled:` line when the run is not moving. It names the cause and gives the evidence: an orphaned owner (when it last ticked), a granted start the loop does not take (its last `loop on` reply), overdue planning (what the plan files lack), or a shipping error that repeats (its note and count). - A run whose owning session has gone quiet for 10 minutes is reported as orphaned in other sessions, and `/auto-claude resume` picks it up there. - Only the owning session receives the unattended rules. diff --git a/auto-claude/hooks/auto.test.ts b/auto-claude/hooks/auto.test.ts index a548243..5a2ec22 100644 --- a/auto-claude/hooks/auto.test.ts +++ b/auto-claude/hooks/auto.test.ts @@ -5,7 +5,7 @@ import { countSarif, parseDeploy } from './deploy' import { LOOP_PROTOCOL } from './loop-protocol' import type { ProtocolStep } from './loop-protocol' import { finalCheckOf, fixLine, kickoffText, parseQueue, validatePlan } from './plan' -import { capReached, inFlight, mergeable, parseChecks, prNumber, readLog } from './ship' +import { capReached, idleDue, inFlight, mergeable, parseChecks, prNumber, progressSig, readLog, repoSlug } from './ship' type On = Parameters[1] type Engine = Parameters[0] @@ -59,19 +59,60 @@ describe('ship', () => { test('log tail reads from the offset and survives a trimmed log', () => { const log = '{"event":"start","branch":"goal/01-a"}\n{"event":"met","goal":"a","branch":"goal/01-a"}\nnot json\n' expect(readLog(log, 0).entries.map(e => e.event)).toEqual(['start', 'met']) - expect(readLog(log, 2)).toEqual({ entries: [], offset: 3 }) + // A last line that does not parse may be torn: the cursor stays before it. + expect(readLog(log, 2)).toEqual({ entries: [], offset: 2, last: '{"event":"met","goal":"a","branch":"goal/01-a"}' }) expect(readLog(log, 99).entries).toHaveLength(2) }) + test('the log is followed by its last line read, so a trimmed log neither stalls nor replays', () => { + const line = (i: number) => JSON.stringify({ ts: `t${i}`, event: 'start', branch: `b${i}` }) + const text = (from: number, to: number) => `${Array.from({ length: to - from }, (_, k) => line(from + k)).join('\n')}\n` + const first = readLog(text(0, 5), 0) + expect(first.last).toBe(line(4)) + // Trimmed to the same length: only the two new lines. + expect(readLog(text(2, 7), first.offset, first.last).entries.map(e => e.branch)).toEqual(['b5', 'b6']) + // Cleared by hand and refilled: nothing read before is read again. + expect(readLog(text(0, 5), 5, line(4)).entries).toEqual([]) + // Trimmed past the last line read (goal-mode keeps the length, so it is + // never shorter than what was read): the whole window is new. + expect(readLog(text(10, 15), 5, line(4)).entries).toHaveLength(5) + }) + + test('a log read while goal-mode rewrites it (a prefix, a torn last line) replays nothing and loses nothing', () => { + const line = (i: number) => JSON.stringify({ ts: `t${i}`, event: 'start', branch: `b${i}` }) + const text = (from: number, to: number) => `${Array.from({ length: to - from }, (_, k) => line(from + k)).join('\n')}\n` + // A prefix of a log already read to line 4: nothing, and the cursor stays. + const prefix = readLog(text(0, 3), 5, line(4)) + expect(prefix.entries).toEqual([]) + expect(prefix.last).toBe(line(4)) + // A torn last line is not read, and the cursor stays on the last whole one. + const full = text(0, 6) + const torn = readLog(full.slice(0, full.length - 10), 5, line(4)) + expect(torn.entries).toEqual([]) + expect(torn.last).toBe(line(4)) + // The next read, whole, yields that line once. + expect(readLog(full, torn.offset, torn.last).entries.map(e => e.branch)).toEqual(['b5']) + }) + test('checks, mergeability and PR numbers', () => { expect(parseChecks('[{"bucket":"pass"},{"bucket":"skipping"}]')).toBe('green') expect(parseChecks('[{"bucket":"pass"},{"bucket":"pending"}]')).toBe('pending') expect(parseChecks('[{"bucket":"fail"}]')).toBe('red') expect(parseChecks('')).toBe('none') - expect(mergeable('{"state":"OPEN","mergeable":"CONFLICTING"}')).toEqual({ open: true, conflict: true, unknown: false }) + expect(mergeable('{"state":"OPEN","mergeable":"CONFLICTING"}')).toEqual({ open: true, merged: false, conflict: true, unknown: false }) + // A merged PR reports mergeable UNKNOWN; that must not read as "not yet judged". + expect(mergeable('{"state":"MERGED","mergeable":"UNKNOWN"}')).toEqual({ open: false, merged: true, conflict: false, unknown: false }) expect(prNumber('https://github.com/o/r/pull/42\n')).toBe(42) }) + test('gh is aimed at the pushed remote\'s repository, whatever the URL form', () => { + expect(repoSlug('git@github.com:o/r.git\n')).toBe('o/r') + expect(repoSlug('https://github.com/o/r.git')).toBe('o/r') + expect(repoSlug('https://github.com/o/r')).toBe('o/r') + expect(repoSlug('ssh://git@github.com/o/r.git')).toBe('o/r') + expect(repoSlug('https://example.com/o/r.git')).toBeUndefined() + }) + test('one milestone ships at a time; caps stop the run', () => { const s = (state: 'push' | 'ci' | 'merged' | 'held', headRef: string) => ({ goal: headRef, headRef, branch: headRef, state, red: 0, since: 0 }) expect(inFlight([s('merged', 'a'), s('ci', 'b'), s('push', 'c')])?.headRef).toBe('b') @@ -106,6 +147,14 @@ function world(on: On, store: Record = {}) { // GitHub CI mode, and a branch push whose git cannot be spawned. github: false, gitGone: false, + // Whether the directory is a git repository (git init makes it one), and whether it has an origin. + repo: true, + origin: true, + // origin/HEAD as `git symbolic-ref` sees it ('' when it was never set), and the remote's own HEAD branch. + originHead: 'origin/main', + remoteHead: 'main', + // Paths whose writes fail (a read-only file, a full disk). + unwritable: new Set(), // What goal-mode answers to `loop on`. loopReply: 'Loop on: started', // The deploy command's exit, run in the driver's fresh checkout. @@ -135,6 +184,7 @@ function world(on: On, store: Record = {}) { }) on('fs.exists', (_, e) => ({ value: exists(e.path) })) on('fs.write', (_, e) => { + if (w.unwritable.has(e.path)) return { deny: `EACCES ${e.path}` } w.files.set(e.path, e.text) return { value: undefined } }) @@ -152,7 +202,12 @@ function world(on: On, store: Record = {}) { const ok = (stdout = '', exitCode = 0) => ({ value: { exitCode, stdout, stderr: '', isStdoutTruncated: false, isStderrTruncated: false } }) // git cannot start in a directory that is gone (Bun: posix_spawn ENOENT). if (e.init?.cwd !== undefined && removed(e.init.cwd)) return ok('', -1) - if (a.includes('--git-common-dir')) return ok('/repo/.git\n') + if (a.includes('--git-common-dir')) return w.repo ? ok('/repo/.git\n') : ok('', 128) + if (a.includes(' init')) { + w.repo = true + return ok() + } + if (a.includes('symbolic-ref --short HEAD')) return ok('main\n') if (e.argv[0] === 'mv') { const [from, to] = e.argv.slice(-2) as [string, string] const text = w.files.get(from) @@ -161,8 +216,10 @@ function world(on: On, store: Record = {}) { w.files.delete(from) return ok() } - if (a.endsWith(' remote')) return ok('origin\n') - if (a.includes('symbolic-ref')) return ok('origin/main\n') + if (a.endsWith(' remote')) return ok(w.origin ? 'origin\n' : '') + if (a.includes(' remote get-url ')) return ok('git@github.com:o/r.git\n') + if (a.includes('symbolic-ref')) return w.originHead === '' ? ok('', 128) : ok(`${w.originHead}\n`) + if (a.includes('ls-remote --symref')) return ok(`ref: refs/heads/${w.remoteHead}\tHEAD\nabc123\tHEAD\n`) if (a.startsWith('gh repo view')) return ok('', w.github ? 0 : 1) if (a.startsWith('gh pr create')) return ok('https://github.com/o/r/pull/7\n') if (a.startsWith('gh pr checks')) { @@ -192,10 +249,12 @@ function world(on: On, store: Record = {}) { const cmd = ($: Engine, args: string) => $.command.run({ command: 'auto-claude', args, origin: { kind: 'composer' }, presentation: { isFullscreen: false, columns: 80 } }) // goal-mode's side: each start is logged, then spends one of the gate's credit. -const log = (w: ReturnType, lines: Array<{ event: string } & Record>) => { +// cap: keep only the last `cap` lines after each append, as goal-mode does (LOOP_LOG_MAX_LINES). +const log = (w: ReturnType, lines: Array<{ event: string } & Record>, cap?: number) => { for (const line of lines) { const prev = w.files.get('/repo/.claude/goal-loop.log') ?? '' - w.files.set('/repo/.claude/goal-loop.log', `${prev}${JSON.stringify(line)}\n`) + const all = `${prev}${JSON.stringify(line)}\n`.split('\n').filter(l => l !== '') + w.files.set('/repo/.claude/goal-loop.log', `${(cap === undefined ? all : all.slice(-cap)).join('\n')}\n`) if (line.event !== 'start') continue const credit = Number.parseInt(w.files.get('/repo/.claude/goal-loop.gate') ?? '', 10) if (Number.isFinite(credit)) w.files.set('/repo/.claude/goal-loop.gate', `${Math.max(credit - 1, 0)}\n`) @@ -386,6 +445,88 @@ describe('a run, local CI', () => { expect((await cmd($, 'status')).text).toContain('auto-claude off · phase done') }) + test('a loop log at goal-mode\'s 2000-line cap still yields every new event (the line count no longer grows)', async ($, on) => { + const w = world(on) + const CAP = 2000 + w.files.set('/repo/.claude/goal-loop.log', `${Array.from({ length: CAP }, (_, i) => JSON.stringify({ ts: `t${i}`, event: 'denied' })).join('\n')}\n`) + await planned($, w) + await w.clock.advance(30_000) + log(w, [ + { ts: 'a1', event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { ts: 'a2', event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ], CAP) + w.files.set('/repo/.claude/goals.md', QUEUE.replace('- [ ] add', '- [x] add')) + await w.clock.advance(30_000) + expect(w.pushes.some(p => p.includes('HEAD:main') && p.includes('goal-01-add'))).toBe(true) + // The next milestone is read too, after more trimming. + log(w, [ + { ts: 'b1', event: 'start', goal: 'sub() returns a-b', worktree: '/repo/worktrees/goal-02-sub', branch: 'goal/02-sub' }, + { ts: 'b2', event: 'met', goal: 'sub() returns a-b', branch: 'goal/02-sub' }, + ], CAP) + await w.clock.advance(30_000) + expect(w.pushes.some(p => p.includes('HEAD:main') && p.includes('goal-02-sub'))).toBe(true) + // Each milestone landed once: no event was read twice. + expect(w.pushes.filter(p => p.includes('HEAD:main')).length).toBe(2) + expect((await cmd($, 'status')).text).toContain('2 merged, 0 held') + }) + + test('an origin with no origin/HEAD (a clone made by remote add + fetch) takes the remote\'s own default branch, not main', async ($, on) => { + const w = world(on) + w.originHead = '' + w.remoteHead = 'master' + await $.session.start({ cwd: '/repo', surface: 'terminal', isInteractive: true }) + expect((await cmd($, 'start')).text).toContain('merges into master') + expect(w.files.get('/repo/.claude/goal-loop.base')).toBe('origin/master\n') + }) + + test('a tick that fails every time is told to the person once, with its error, and again only after it recovered', async ($, on) => { + const w = world(on) + await planned($, w) + await w.clock.advance(30_000) + log(w, [ + { event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ]) + w.files.set('/repo/.claude/goals.md', QUEUE.replace('- [ ] add', '- [x] add')) + // Landing milestone 1 is fine; granting the next start cannot write the gate. + w.unwritable.add(GATE) + for (let i = 0; i < 10; i++) await w.clock.advance(30_000) + const told = w.notifies.filter(n => n.includes('tick fails')) + expect(told).toHaveLength(1) + expect(told[0]).toContain('EACCES') + expect((await cmd($, 'status')).text).toContain('stalled: every tick fails') + w.unwritable.delete(GATE) + await w.clock.advance(30_000) + expect((await cmd($, 'status')).text).not.toContain('every tick fails') + }) + + test('the time cap is counted on the engine\'s clock: 24 h after the start the run stops', async ($, on) => { + const w = world(on) + await planned($, w) + log(w, [{ event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }]) + await w.clock.advance(23 * 60 * MIN) + expect(w.notifies.some(n => n.includes('time cap'))).toBe(false) + await w.clock.advance(61 * MIN) + expect(w.notifies.some(n => n.includes('stopped: the time cap (24 h) was reached'))).toBe(true) + }) + + test('a loop log cleared by hand is read from its start once it stays short for a second tick', async ($, on) => { + const w = world(on) + w.files.set('/repo/.claude/goal-loop.log', `${Array.from({ length: 40 }, (_, i) => JSON.stringify({ ts: `t${i}`, event: 'denied' })).join('\n')}\n`) + await planned($, w) + await w.clock.advance(30_000) + w.files.set('/repo/.claude/goal-loop.log', '') + log(w, [ + { ts: 'a1', event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { ts: 'a2', event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ]) + w.files.set('/repo/.claude/goals.md', QUEUE.replace('- [ ] add', '- [x] add')) + await w.clock.advance(30_000) + expect(w.pushes.some(p => p.includes('goal-01-add'))).toBe(false) + await w.clock.advance(30_000) + expect(w.pushes.filter(p => p.includes('HEAD:main') && p.includes('goal-01-add'))).toHaveLength(1) + }) + test('the gate stays shut while the loop is still building a milestone', async ($, on) => { const w = world(on) await $.session.start({ cwd: '/repo', surface: 'terminal', isInteractive: true }) @@ -574,6 +715,9 @@ describe('a run, local CI', () => { expect(w.merged).toHaveLength(1) expect(w.files.get(GATE)).toBe('1\n') expect(w.commands).toEqual(['goal-mode loop on', 'goal-mode loop on']) + // Every gh call names origin's repository, so a fork's upstream is never picked. + for (const call of w.runs.filter(a => /^gh pr (create|checks|view|merge)/.test(a))) expect(call).toContain('--repo o/r') + expect(w.runs.some(a => a.startsWith('gh pr create'))).toBe(true) // The next one's CI is pending: looked at again within 10 s, landed once green. log(w, [ @@ -615,7 +759,7 @@ describe('a run, local CI', () => { expect(w.files.get(PIECE)).toBe('') await cmd($, 'start') expect(w.files.get(PIECE)).toContain('auto·R1·plan') - await $.session.end({ reason: 'clear', sessionId: 'me' }) + await $.session.end({ reason: 'clear', sessionId: 'me', resume: { id: 'me' } }) expect(w.files.get(PIECE)).toBe('') }) @@ -764,3 +908,151 @@ describe('a run, local CI', () => { }) }) + +describe('a directory with no git repository', () => { + test('start runs git init, ignores secrets, makes the first commit, and lands milestones in main itself', async ($, on) => { + const w = world(on) + w.repo = false + w.origin = false + await $.session.start({ cwd: '/repo', surface: 'terminal', isInteractive: true }) + const text = (await cmd($, 'start build a tiny lib')).text ?? '' + expect(text).toContain('auto-claude on in /repo') + expect(text).toContain('Initialized a git repository') + expect(text).toContain('No remote: milestones land in main') + expect(w.runs.some(a => a.includes(' init -b main'))).toBe(true) + expect(w.runs.some(a => a.includes('commit --allow-empty -m Initial commit'))).toBe(true) + expect(w.runs.some(a => a.includes('config receive.denyCurrentBranch updateInstead'))).toBe(true) + expect(w.files.get('/repo/.gitignore')).toMatch(/\.env[\s\S]*worktrees\//) + // The loop builds from the local branch, with no remote to fetch. + expect(w.files.get('/repo/.claude/goal-loop.base')).toBe('main\n') + await w.clock.advance(200) + expect(w.prompts[0]).toContain('This repository has no remote') + + w.files.set('/repo/.claude/auto-plan.md', PLAN) + w.files.set('/repo/.claude/goals.md', QUEUE) + await $.tool.call({ tool: 'mcp__auto-claude__auto_plan_ready' } as never) + await w.clock.advance(30_000) + log(w, [ + { event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ]) + await w.clock.advance(30_000) + expect(w.pushes.some(a => a.endsWith(' push . HEAD:main'))).toBe(true) + expect(w.runs.some(a => a.includes(' fetch '))).toBe(false) + expect(w.runs.some(a => a.includes('rebase main'))).toBe(true) + }) + + test('the home directory is not bootstrapped', async ($, on) => { + const w = world(on) + w.repo = false + w.cwd = '/home' + await $.session.start({ cwd: '/home', surface: 'terminal', isInteractive: true }) + expect((await cmd($, 'start')).text).toContain('Not a project directory') + expect(w.runs.some(a => a.includes(' init'))).toBe(false) + }) +}) + +describe('an idle run', () => { + const nudges = (w: ReturnType) => w.notifies.filter(n => n.includes('(nudge ')) + + test('progressSig ignores an error note; idleDue is 15 min, then 30 min apart, at most 3', () => { + const ship = { goal: 'g', headRef: 'a', branch: 'a', state: 'push' as const, red: 0, since: 0 } + const base = { started: 1, finished: 1, merges: 0, held: 0 } + expect(progressSig({ ...base, ships: [ship] })).toBe(progressSig({ ...base, ships: [{ ...ship, note: 'push failed' }] })) + expect(progressSig({ ...base, ships: [ship] })).not.toBe(progressSig({ ...base, ships: [{ ...ship, state: 'ci' as const }] })) + const r = { on: true, phase: 'run' as const, movedAt: 0 } + const min = (n: number) => n * 60_000 + expect(idleDue({ ...r }, min(14))).toBeUndefined() + expect(idleDue({ ...r }, min(15))).toBe(1) + expect(idleDue({ ...r, idleNudges: 1, idleNudgedAt: min(15) }, min(44))).toBeUndefined() + expect(idleDue({ ...r, idleNudges: 1, idleNudgedAt: min(15) }, min(45))).toBe(2) + expect(idleDue({ ...r, idleNudges: 3, idleNudgedAt: min(75) }, min(500))).toBeUndefined() + expect(idleDue({ ...r, phase: 'plan' }, min(500))).toBeUndefined() + }) + + test('a loop that does not start is nudged at 15 min, then every 30 min, three times, and asked again each time', async ($, on) => { + const w = world(on) + w.loopReply = 'A session goal is active ("x"): /goal-mode clear it before starting the loop.' + await planned($, w) + for (let i = 0; i < 28; i++) await w.clock.advance(30_000) + expect(nudges(w)).toHaveLength(0) + await w.clock.advance(2 * MIN) + expect(nudges(w)).toHaveLength(1) + expect(nudges(w)[0]).toContain('nudge 1/3') + expect(nudges(w)[0]).toContain('A session goal is active') + const asks = w.commands.filter(c => c === 'goal-mode loop on').length + // The next nudge is 30 min later, not sooner; the third is the last. + for (let i = 0; i < 58; i++) await w.clock.advance(30_000) + expect(nudges(w)).toHaveLength(1) + await w.clock.advance(MIN) + expect(nudges(w)).toHaveLength(2) + expect(w.commands.filter(c => c === 'goal-mode loop on').length).toBeGreaterThan(asks) + for (let i = 0; i < 70; i++) await w.clock.advance(MIN) + expect(nudges(w)).toHaveLength(3) + expect(nudges(w)[2]).toContain('No more nudges') + expect((await cmd($, 'status')).text).toContain('stalled:') + }) + + test('progress starts the count over; a goal being built and a pending CI are not idle', async ($, on) => { + const w = world(on) + w.github = true + await planned($, w) + await w.clock.advance(31 * MIN) + expect(nudges(w)).toHaveLength(1) + + // A goal started and being built for an hour: waiting, not idle. + log(w, [{ event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }]) + for (let i = 0; i < 4; i++) await w.clock.advance(15 * MIN) + expect(nudges(w)).toHaveLength(1) + + // Met, its CI pending with nothing wrong: waiting too. + w.checks = '[{"bucket":"pending"}]' + log(w, [{ event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }]) + for (let i = 0; i < 4; i++) await w.clock.advance(15 * MIN) + expect(nudges(w)).toHaveLength(1) + + // Merged: progress. The count starts again at nudge 1 once the loop sits idle. + w.checks = '[{"bucket":"pass"}]' + await w.clock.advance(30_000) + expect(w.merged).toHaveLength(1) + w.loopReply = 'A session goal is active ("x")' + await w.clock.advance(31 * MIN) + expect(nudges(w).some(n => n.includes('nudge 1/3') && n !== nudges(w)[0])).toBe(true) + }) + + test('a CI that stays pending (no runner takes it) or a mergeability GitHub never decides is waiting only 90 min, then nudged', async ($, on) => { + const w = world(on) + w.github = true + w.checks = '[{"bucket":"pending"}]' + await planned($, w) + log(w, [ + { event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ]) + for (let i = 0; i < 6; i++) await w.clock.advance(15 * MIN) + expect(nudges(w)).toHaveLength(0) + for (let i = 0; i < 4; i++) await w.clock.advance(5 * MIN) + const n = nudges(w) + expect(n).toHaveLength(1) + expect(n[0]).toContain('PR #7') + expect(n[0]).toContain('min for CI') + expect(n[0]).not.toContain('Asking the loop to start again') + expect((await cmd($, 'status')).text).toContain('stalled: PR #7 has waited') + }) + + test('a shipping error that does not clear is nudged with its note, and the loop is not asked', async ($, on) => { + const w = world(on) + w.github = true + w.gitGone = true + await planned($, w) + log(w, [ + { event: 'start', goal: 'add() returns a+b', worktree: '/repo/worktrees/goal-01-add', branch: 'goal/01-add' }, + { event: 'met', goal: 'add() returns a+b', branch: 'goal/01-add' }, + ]) + for (let i = 0; i < 34; i++) await w.clock.advance(30_000) + const n = nudges(w) + expect(n).toHaveLength(1) + expect(n[0]).toContain('ENOENT') + expect(n[0]).not.toContain('Asking the loop to start again') + }) +}) diff --git a/auto-claude/hooks/plan.ts b/auto-claude/hooks/plan.ts index c55107c..7e98d31 100644 --- a/auto-claude/hooks/plan.ts +++ b/auto-claude/hooks/plan.ts @@ -51,7 +51,7 @@ export function validatePlan(plan: string | undefined, queue: string | undefined export const READY_TOOL = 'auto_plan_ready' -export function kickoffText(hint: string, base: string): string { +export function kickoffText(hint: string, base: string, noRemote = false): string { return [ '[auto-claude] The person started an autonomous run of this repository. Plan it now; build nothing yet.', hint !== '' ? `Their aim: ${hint}` : 'No aim was given: finish what this repository is evidently for.', @@ -61,6 +61,7 @@ export function kickoffText(hint: string, base: string): string { `2. .claude/goals.md: one line per milestone, in order: \`- [ ] :: check: \`. Each milestone is one reviewable PR, built in its own worktree from ${base} after the earlier milestones are merged, so each check must pass from such a checkout. State only what is true on the milestone's own branch: never \"merged\", \"pushed\" or \"PR opened\" (that happens after it is met).`, `Then call mcp__auto-claude__${READY_TOOL}. It checks the files and starts the run: goal-mode's loop builds each milestone, and auto-claude pushes it, opens a PR, waits for CI and merges when green.`, 'Nobody is watching: decide every choice yourself (the recommended or safest option), write each decision with its reason under ## Decisions, and ask only for credentials, payment or an irreversible step.', + ...(noRemote ? [`This repository has no remote: each milestone lands in ${base} in this checkout, judged by its own check, with no PR and no CI. If the repository is empty or nearly so, make the first milestone its scaffold (README, manifest, test setup, one passing test) with a check that proves it runs.`] : []), ].join('\n') } diff --git a/auto-claude/hooks/register.ts b/auto-claude/hooks/register.ts index f28d854..f4814e3 100644 --- a/auto-claude/hooks/register.ts +++ b/auto-claude/hooks/register.ts @@ -5,7 +5,8 @@ import { fixLine, kickoffText, parseQueue, planNudgeText, POLICY, READY_TOOL, re import { segment } from './segment' import { codeqlSuite, countSarif, DEPLOY_FILE, parseDeploy } from './deploy' import type { DeploySpec } from './deploy' -import { capReached, diagnose, gateCredit, inFlight, MAX_RED, mergeable, parseChecks, prNumber, readLog, summary } from './ship' +import { capReached, diagnose, gateCredit, idleDue, inFlight, isWaiting, MAX_IDLE_NUDGES, MAX_RED, mergeable, parseChecks, prNumber, progressSig, readLog, repoSlug, summary } from './ship' +import type { StallFacts } from './ship' import type { Caps } from './ship' // auto-claude: an unattended run of one repository, driven on top of @@ -17,6 +18,8 @@ import type { Caps } from './ship' // $.process.run, never as a model turn, and the model is denied merging. const COMMAND = 'auto-claude' +// The ready tool's full name, as the tool.call matcher sees it. +const READY_TOOL_ID = 'mcp__auto-claude__auto_plan_ready' const STORE = (root: string) => `auto:${root}` const FILES = (root: string) => ({ plan: `${root}/.claude/auto-plan.md`, @@ -56,6 +59,11 @@ let fastTick = false let activeRoot: string | undefined // The owner an orphaned run was last reported for, once per session. let orphanTold: string | undefined +// Ticks failed in a row and the last error: a tick that throws every time +// (an unwritable gate, a broken store) moves nothing and must not stay silent. +let tickFails = 0 +let tickError: string | undefined +const TICK_FAIL_TELL = 3 type Run = { exitCode: number; stdout: string; stderr: string } @@ -70,6 +78,13 @@ async function run($: EngineInterface, argv: string[], cwd: string, timeoutMs = const out = (r: Run) => (r.stderr.trim() || r.stdout.trim()).slice(0, 400) +// A repository with no remote lands its milestones in its own default branch: its "remote" is itself. +const LOCAL = '.' +const refOf = (r: { remote: string; defaultBranch: string }, branch = r.defaultBranch) => (r.remote === LOCAL ? branch : `${r.remote}/${branch}`) +async function fetchRemote($: EngineInterface, r: { remote: string }, cwd: string): Promise { + if (r.remote !== LOCAL) await run($, ['git', '-C', cwd, 'fetch', '-q', r.remote], cwd) +} + async function readText($: EngineInterface, path: string): Promise { try { return await $.fs.read(path) @@ -158,7 +173,7 @@ async function release($: EngineInterface, r: AutoRecord, pollMs: number): Promi // goal-mode logs a start before it spends the credit: the gate is read // first and the log after, so a spent gate is never mistaken for an idle loop. const credit = gateCredit(await readText($, f.gate)) ?? 0 - if (readLog((await readText($, f.log)) ?? '', r.logOffset).entries.some(e => e.event === 'start')) return r + if (readLog((await readText($, f.log)) ?? '', r.logOffset, r.logLast).entries.some(e => e.event === 'start')) return r if (credit <= 0) { await $.fs.write(f.gate, '1\n') const next: AutoRecord = { ...r, want: 'on' } @@ -259,13 +274,16 @@ async function advance($: EngineInterface, r: AutoRecord, item: ShipItem): Promi return { r: { ...r, held: r.held + 1 }, item: update({ state: 'held', note: where.gone }) } } const wt = where.wt + // gh is told which repository the branch was pushed to (see repoSlug). + const slug = r.remote === LOCAL ? undefined : repoSlug((await run($, ['git', '-C', wt, 'remote', 'get-url', r.remote], wt, 15_000)).stdout) + const repo = slug === undefined ? [] : ['--repo', slug] if (item.state === 'push') { if (r.ciMode === 'local') return { r, item: update({ state: 'ci' }) } const pushed = await run($, ['git', '-C', wt, 'push', '-u', r.remote, `${item.branch}:${item.headRef}`, ...(item.branch !== item.headRef ? ['--force-with-lease'] : [])], wt) if (pushed.exitCode !== 0) return { r, item: update({ note: `push failed: ${out(pushed)}` }) } if (item.pr !== undefined) return { r, item: update({ state: 'ci', since: await $.clock.now() }) } - const made = await run($, ['gh', 'pr', 'create', '--head', item.headRef, '--base', r.defaultBranch, '--title', item.goal.slice(0, 120), '--body', `Milestone of an auto-claude run.\n\nCheck: \`${item.check ?? '-'}\`\n\nPlan: .claude/auto-plan.md (local to the run).`], wt) + const made = await run($, ['gh', 'pr', 'create', ...repo, '--head', item.headRef, '--base', r.defaultBranch, '--title', item.goal.slice(0, 120), '--body', `Milestone of an auto-claude run.\n\nCheck: \`${item.check ?? '-'}\`\n\nPlan: .claude/auto-plan.md (local to the run).`], wt) const pr = prNumber(made.stdout + made.stderr) if (pr === undefined) return { r, item: update({ note: `gh pr create failed: ${out(made)}` }) } await tell($, r, `opened PR #${pr}: ${item.goal.slice(0, 80)}`) @@ -276,8 +294,8 @@ async function advance($: EngineInterface, r: AutoRecord, item: ShipItem): Promi let failure: string | undefined if (r.ciMode === 'local') { - await run($, ['git', '-C', wt, 'fetch', '-q', r.remote], wt) - const rebased = await run($, ['git', '-C', wt, 'rebase', `${r.remote}/${r.defaultBranch}`], wt) + await fetchRemote($, r, wt) + const rebased = await run($, ['git', '-C', wt, 'rebase', refOf(r)], wt) if (rebased.exitCode !== 0) { await run($, ['git', '-C', wt, 'rebase', '--abort'], wt) failure = `does not rebase onto ${r.defaultBranch}: ${out(rebased)}` @@ -291,7 +309,7 @@ async function advance($: EngineInterface, r: AutoRecord, item: ShipItem): Promi } } } else { - const checks = parseChecks((await run($, ['gh', 'pr', 'checks', String(item.pr), '--json', 'bucket'], wt)).stdout) + const checks = parseChecks((await run($, ['gh', 'pr', 'checks', String(item.pr), ...repo, '--json', 'bucket'], wt)).stdout) if (checks === 'pending') return { r, item } // Asked right after the PR was opened, GitHub has usually not registered // its checks yet: that is pending too, until the grace runs out. @@ -299,17 +317,20 @@ async function advance($: EngineInterface, r: AutoRecord, item: ShipItem): Promi if (checks === 'red') failure = 'CI failed (gh pr checks)' if (failure === undefined) { // Looked at again right before merging. - const state = mergeable((await run($, ['gh', 'pr', 'view', String(item.pr), '--json', 'state,mergeable'], wt)).stdout) + const state = mergeable((await run($, ['gh', 'pr', 'view', String(item.pr), ...repo, '--json', 'state,mergeable'], wt)).stdout) if (state.unknown) return { r, item } - if (!state.open) return { r, item: update({ state: 'held', note: 'the PR is no longer open' }) } - if (state.conflict) failure = `conflicts with ${r.defaultBranch}` + // A merge whose reply was lost (a 502 after GitHub merged it) has landed. + if (state.merged) { + // Fall through to the merged bookkeeping below. + } else if (!state.open) return { r, item: update({ state: 'held', note: 'the PR is no longer open' }) } + else if (state.conflict) failure = `conflicts with ${r.defaultBranch}` else { // CI judged the PR; the milestone's own check is judged on the pushed head too. await run($, ['git', '-C', r.root, 'fetch', '-q', r.remote, item.headRef], r.root) failure = await verifyFresh($, r, item, `${r.remote}/${item.headRef}`) } - if (failure === undefined) { - const merged = await run($, ['gh', 'pr', 'merge', String(item.pr), '--squash', '--delete-branch'], wt) + if (failure === undefined && !state.merged) { + const merged = await run($, ['gh', 'pr', 'merge', String(item.pr), ...repo, '--squash', '--delete-branch'], wt) if (merged.exitCode !== 0) return { r, item: update({ note: `merge failed: ${out(merged)}` }) } } } @@ -352,9 +373,9 @@ async function advance($: EngineInterface, r: AutoRecord, item: ShipItem): Promi async function deployMerged($: EngineInterface, r: AutoRecord, spec: DeploySpec): Promise { const dir = `${r.root}/worktrees/auto-deploy` await run($, ['git', '-C', r.root, 'worktree', 'remove', '--force', dir], r.root) - await run($, ['git', '-C', r.root, 'fetch', '-q', r.remote], r.root) - const made = await run($, ['git', '-C', r.root, 'worktree', 'add', '--detach', dir, `${r.remote}/${r.defaultBranch}`], r.root) - if (made.exitCode !== 0) return `no checkout of ${r.remote}/${r.defaultBranch}: ${out(made)}` + await fetchRemote($, r, r.root) + const made = await run($, ['git', '-C', r.root, 'worktree', 'add', '--detach', dir, refOf(r)], r.root) + if (made.exitCode !== 0) return `no checkout of ${refOf(r)}: ${out(made)}` try { for (const language of spec.codeql) { const blocked = await codeqlGate($, r, dir, language) @@ -393,8 +414,8 @@ async function finalCheck($: EngineInterface, r: AutoRecord, caps: Caps): Promis // Judged on what was merged: a fresh detached checkout of the default branch. const dir = `${r.root}/worktrees/auto-final` await run($, ['git', '-C', r.root, 'worktree', 'remove', '--force', dir], r.root) - await run($, ['git', '-C', r.root, 'fetch', '-q', r.remote], r.root) - const made = await run($, ['git', '-C', r.root, 'worktree', 'add', '--detach', dir, `${r.remote}/${r.defaultBranch}`], r.root) + await fetchRemote($, r, r.root) + const made = await run($, ['git', '-C', r.root, 'worktree', 'add', '--detach', dir, refOf(r)], r.root) const ran = made.exitCode === 0 ? await run($, ['bash', '-lc', check], dir, 600_000) : made await run($, ['git', '-C', r.root, 'worktree', 'remove', '--force', dir], r.root) if (ran.exitCode === 0) { @@ -415,6 +436,7 @@ async function finalCheck($: EngineInterface, r: AutoRecord, caps: Caps): Promis async function tick($: EngineInterface, caps: Caps, pollMs: number): Promise { if (ticking || activeRoot === undefined) return ticking = true + let failed = false try { const got = await readRecord($, activeRoot) if (got === undefined) return @@ -437,7 +459,8 @@ async function tick($: EngineInterface, caps: Caps, pollMs: number): Promise= 1) read = readLog(logText, 0) + const { entries, offset, last } = read + r = { ...r, logOffset: offset, logLast: last, logHeld: read.held === true ? (r.logHeld ?? 0) + 1 : 0 } for (const e of entries) { if (e.event === 'start' && e.branch !== undefined) { r = { ...r, started: r.started + 1, worktrees: { ...r.worktrees, [e.branch]: e.worktree ?? '' }, kicks: 0, kickTold: false, kickReply: undefined, ...(r.kickTold === true ? { note: undefined } : {}) } @@ -509,6 +537,7 @@ async function tick($: EngineInterface, caps: Caps, pollMs: number): Promise {}) + } } finally { + if (!failed) { + tickFails = 0 + tickError = undefined + } ticking = false } } @@ -556,18 +596,89 @@ async function clearStop($: EngineInterface, root: string): Promise<{ cleared: b const clearedNote = (c: { cleared: boolean }) => (c.cleared ? ' A leftover .claude/auto-claude.stop was moved to .claude/auto-claude.stop.cleared.' : '') +// Lines a bootstrapped repository ignores before its first commit: secrets and +// noise must not enter a history auto-claude then builds on. +const BOOTSTRAP_IGNORE = ['.env', '.env.*', '!.env.example', '*.pem', '*.key', '*.p12', '*.pfx', 'node_modules/', '.DS_Store', 'worktrees/', '.claude/settings.local.json'] +// A directory with more files than this, or other repositories inside it, is not one project. +const BOOTSTRAP_MAX_FILES = 3000 + +// A directory that is not a git repository (or has one with no commit yet) is +// made into one: `git init` on main, a .gitignore for secrets, one initial +// commit of what is there. Returns why it will not, for a directory that is +// plainly not a single project (the home directory, a folder of repositories). +async function bootstrapRepo($: EngineInterface, dir: string, hasRepo: boolean): Promise<{ root: string; note: string } | { error: string }> { + const home = (await $.env.get('HOME').catch(() => undefined))?.replace(/\/$/, '') + if (dir === '/' || dir === home) return { error: `Not a project directory: ${dir}. cd into the project's folder and start there.` } + const nested = (await run($, ['bash', '-c', 'find . -mindepth 2 -maxdepth 3 -name .git -not -path "./worktrees/*" | head -1'], dir, 30_000)).stdout.trim() + if (nested !== '') return { error: `Not bootstrapped: ${dir} holds other repositories (${nested}); start inside one of them.` } + const counted = (await run($, ['bash', '-c', `find . -type f -not -path "./.git/*" -not -path "*/node_modules/*" | head -${BOOTSTRAP_MAX_FILES + 1} | wc -l`], dir, 60_000)).stdout.trim() + if (Number(counted) > BOOTSTRAP_MAX_FILES) return { error: `Not bootstrapped: ${dir} holds more than ${BOOTSTRAP_MAX_FILES} files, so it is not one project; start in the project's folder, or run git init yourself.` } + if (!hasRepo) { + let init = await run($, ['git', '-C', dir, 'init', '-b', 'main'], dir, 30_000) + // git before 2.28 has no -b. + if (init.exitCode !== 0) { + init = await run($, ['git', '-C', dir, 'init'], dir, 30_000) + if (init.exitCode === 0) await run($, ['git', '-C', dir, 'symbolic-ref', 'HEAD', 'refs/heads/main'], dir, 15_000) + } + if (init.exitCode !== 0) return { error: `Not bootstrapped: git init failed in ${dir}: ${out(init)}` } + } + const root = await repoRoot($, dir) + if (root === undefined) return { error: `Not bootstrapped: ${dir} is still not a git repository after git init.` } + await ensureLines($, `${root}/.gitignore`, BOOTSTRAP_IGNORE, '# auto-claude bootstrap') + await run($, ['git', '-C', root, 'add', '-A'], root, 60_000) + const staged = (await run($, ['git', '-C', root, 'diff', '--cached', '--name-only'], root, 30_000)).stdout.split('\n').filter(l => l.trim() !== '').length + const made = await run($, ['git', '-C', root, 'commit', '--allow-empty', '-m', 'Initial commit'], root, 60_000) + if (made.exitCode !== 0) return { error: `Not bootstrapped: the initial commit failed (is git user.name / user.email set?): ${out(made)}` } + return { root, note: `${hasRepo ? 'Made the first commit' : 'Initialized a git repository'} here (main, ${staged} file${staged === 1 ? '' : 's'}; secrets and noise are in .gitignore).` } +} + async function startCommand($: EngineInterface, hint: string, ciMode: string): Promise { const where = await sessionRoot($) - const root = where.root - if (root === undefined) return notInRepo(where) + let root = where.root + const notes: string[] = [] + // Not a repository: the session's own directory becomes one (never an + // ancestor found by walking up from a removed directory). + if (root === undefined) { + if (where.used !== where.cwd) return notInRepo(where) + const made = await bootstrapRepo($, where.cwd, false) + if ('error' in made) return made.error + root = made.root + notes.push(made.note) + } const existing = await readRecord($, root) if (existing?.on === true) return `A run is already on here (phase ${existing.phase}); /${COMMAND} status, or /${COMMAND} stop first.` - const remote = (await run($, ['git', '-C', root, 'remote'], root)).stdout.split('\n').map(s => s.trim()).find(s => s === 'origin') ?? undefined - if (remote === undefined) return 'This repository has no `origin` remote: auto-claude ships milestones there.' - await run($, ['git', '-C', root, 'fetch', '-q', remote], root) - const head = (await run($, ['git', '-C', root, 'symbolic-ref', '--short', `refs/remotes/${remote}/HEAD`], root)).stdout.trim() - const defaultBranch = head.startsWith(`${remote}/`) ? head.slice(remote.length + 1) : 'main' - const mode = ciMode === 'github' || ciMode === 'local' ? ciMode : (await run($, ['gh', 'repo', 'view', '--json', 'name'], root, 30_000)).exitCode === 0 ? 'github' : 'local' + // A repository with no commit has nothing to branch milestones from. + if ((await run($, ['git', '-C', root, 'rev-parse', '--verify', '--quiet', 'HEAD'], root, 15_000)).exitCode !== 0) { + const made = await bootstrapRepo($, root, true) + if ('error' in made) return made.error + notes.push(made.note) + } + const hasOrigin = (await run($, ['git', '-C', root, 'remote'], root)).stdout.split('\n').some(s => s.trim() === 'origin') + let remote = 'origin' + let defaultBranch = 'main' + let mode: 'github' | 'local' = 'local' + if (hasOrigin) { + await run($, ['git', '-C', root, 'fetch', '-q', remote], root) + const head = (await run($, ['git', '-C', root, 'symbolic-ref', '--short', `refs/remotes/${remote}/HEAD`], root)).stdout.trim() + // origin/HEAD is set only by clone: a repository whose origin was added by + // hand has none, and its default may well be master. The remote says. + const asked = head.startsWith(`${remote}/`) ? undefined : (await run($, ['git', '-C', root, 'ls-remote', '--symref', remote, 'HEAD'], root, 30_000)).stdout.match(/^ref: refs\/heads\/(\S+)\s+HEAD/m)?.[1] + defaultBranch = head.startsWith(`${remote}/`) ? head.slice(remote.length + 1) : (asked ?? 'main') + mode = ciMode === 'github' || ciMode === 'local' ? ciMode : (await run($, ['gh', 'repo', 'view', '--json', 'name'], root, 30_000)).exitCode === 0 ? 'github' : 'local' + } else { + // No origin: milestones land in this repository's own branch, checked by + // their own checks. A push into the checked-out branch updates its files + // in place, which needs a clean tree. + if (ciMode === 'github') return 'Not started: github CI needs an `origin` remote, and this repository has none.' + remote = LOCAL + const branch = (await run($, ['git', '-C', root, 'symbolic-ref', '--short', 'HEAD'], root, 15_000)).stdout.trim() + if (branch === '') return 'Not started: HEAD is detached, so there is no branch for milestones to land in; check one out.' + defaultBranch = branch + const dirty = (await run($, ['git', '-C', root, 'status', '--porcelain', '--untracked-files=no'], root, 30_000)).stdout.trim() + if (dirty !== '') return `Not started: ${branch} has uncommitted changes, and milestones land in it in place; commit or stash them first.` + await run($, ['git', '-C', root, 'config', 'receive.denyCurrentBranch', 'updateInstead'], root, 15_000) + notes.push(`No remote: milestones land in ${branch} here, checked by their own checks.`) + } // Read once: what this run deploys cannot change while it runs. const deployText = await readText($, `${root}/${DEPLOY_FILE}`) @@ -575,11 +686,13 @@ async function startCommand($: EngineInterface, hint: string, ciMode: string): P if (deploy !== undefined && 'error' in deploy) return `Not started: ${deploy.error}.` const f = FILES(root) + const logLines = ((await readText($, f.log)) ?? '').split('\n').filter(l => l.trim() !== '') + await run($, ['mkdir', '-p', `${root}/.git/info`], root, 15_000) await ensureLines($, f.exclude, EXCLUDES, '# auto-claude') const stop = await clearStop($, root) if ('error' in stop) return `Not started: ${stop.error}.` await ensureLines($, f.deny, DENY, '# auto-claude: merging and releases are the driver\'s') - await $.fs.write(f.base, `${remote}/${defaultBranch}\n`) + await $.fs.write(f.base, `${refOf({ remote, defaultBranch })}\n`) // Closed until the plan is ready. const r: AutoRecord = { on: true, @@ -588,11 +701,12 @@ async function startCommand($: EngineInterface, hint: string, ciMode: string): P sessionId: await $.session.id(), hint, round: 1, - startedAt: Date.now(), + startedAt: await $.clock.now(), remote, defaultBranch, ciMode: mode, - logOffset: ((await readText($, f.log)) ?? '').split('\n').filter(l => l.trim() !== '').length, + logOffset: logLines.length, + logLast: logLines.at(-1), started: 0, finished: 0, credit: true, @@ -610,31 +724,60 @@ async function startCommand($: EngineInterface, hint: string, ciMode: string): P await $.fs.write(f.gate, '0\n') activeRoot = root await writeRecord($, r) - const text = `${kickoffText(hint, `${remote}/${defaultBranch}`)}${deploy === undefined ? '' : `\n- auto-claude deploys after each merge (${DEPLOY_FILE}). Plan no deploy milestone and never deploy yourself.`}` + const text = `${kickoffText(hint, refOf({ remote, defaultBranch }), remote === LOCAL)}${deploy === undefined ? '' : `\n- auto-claude deploys after each merge (${DEPLOY_FILE}). Plan no deploy milestone and never deploy yourself.`}` $.clock.after(100, () => { $.prompt.submit({ text }).catch(() => $.ui.toast('auto-claude: the planning prompt was not submitted; send any message to start.')) }) const moved = where.used === where.cwd ? '' : ` (the session's directory ${where.cwd} no longer exists; used ${where.used})` const deploys = deploy === undefined ? '' : `, deploys after each merge with \`${deploy.command}\`${deploy.codeql.length > 0 ? ` after a CodeQL scan (${deploy.codeql.join(', ')})` : ''}` - return `auto-claude on in ${root}${moved} (${mode} CI, merges into ${defaultBranch}${deploys}). Claude plans the run now.${clearedNote(stop)}` + return `auto-claude on in ${root}${moved} (${mode} CI, merges into ${defaultBranch}${deploys}). Claude plans the run now.${clearedNote(stop)}${notes.length > 0 ? ` ${notes.join(' ')}` : ''}` } -async function stallLine($: EngineInterface, r: AutoRecord): Promise { +async function stallFacts($: EngineInterface, r: AutoRecord): Promise { const f = FILES(r.root) const planErrors = r.phase === 'plan' ? ((v => (v.ok ? [] : v.errors))(validatePlan(await readText($, f.plan), await readText($, f.queue), r.round === 1))) : [] - const why = diagnose(r, { - now: await $.clock.now(), + const now = await $.clock.now() + return { + now, sessionId: await $.session.id(), staleMs: STALE_MS, credit: gateCredit(await readText($, f.gate)), planErrors, maxNudges: MAX_NUDGES, planWaitMs: PLAN_NUDGE_MS, - }) + ...(r.movedAt === undefined ? {} : { idleMs: now - r.movedAt }), + ...(tickFails >= TICK_FAIL_TELL && tickError !== undefined ? { tickFails, tickError } : {}), + } +} + +async function stallLine($: EngineInterface, r: AutoRecord): Promise { + const why = diagnose(r, await stallFacts($, r)) return why === undefined ? undefined : `stalled: ${why}` } +// A run that stays idle is told to the person again and again (the shipping +// error's one note after 10 ticks was missed: a run sat 2 h at a PR that would +// not open). Each nudge names the cause and, where the loop is the one that +// is idle, asks it to start again. Progress, or waiting on a clock of its own +// (a goal being built, a PR's CI), starts the count over. +async function idleCheck($: EngineInterface, r: AutoRecord): Promise { + if (!r.on || r.phase !== 'run') return r + const now = await $.clock.now() + const sig = progressSig(r) + if (sig !== r.sig || r.movedAt === undefined || isWaiting(r, now)) return { ...r, sig, movedAt: now, idleNudges: 0, idleNudgedAt: undefined } + const n = idleDue(r, now) + if (n === undefined) return r + const why = diagnose(r, await stallFacts($, r)) ?? `idle ${Math.round((now - r.movedAt) / 60_000)} min` + // A shipping step that fails retries every tick already. When the loop is the idle one, clearing kickedAt + // lets `release` (this same tick) grant and ask at once, past its backoff; the tries and what was told stay. + const item = inFlight(r.ships) + const heal = item === undefined || item.state === 'fixing' + const last = n >= MAX_IDLE_NUDGES ? ' No more nudges; /auto-claude status keeps the stalled line.' : '' + await tell($, r, `idle ${Math.round((now - r.movedAt) / 60_000)} min (nudge ${n}/${MAX_IDLE_NUDGES}): ${why}.${heal ? ' Asking the loop to start again.' : ''} /auto-claude status for detail; /auto-claude resume re-grants and re-asks.${last}`) + return { ...r, idleNudges: n, idleNudgedAt: now, ...(heal ? { kickedAt: undefined } : {}) } +} + async function statusText($: EngineInterface): Promise { const root = activeRoot ?? (await sessionRoot($)).root const r = root === undefined ? undefined : await readRecord($, root) @@ -693,7 +836,11 @@ export const register: Register = (on, options) => { return { ...composed, sections: [...composed.sections, { id: 'auto-claude:policy', text: POLICY, scope: 'session' as const }] } }) - on('tool.call', { tool: `mcp__auto-claude__${READY_TOOL}` }, async $ => { + // The SDK's Matcher over tool.call's input union is too deep for tsc (TS2589); + // a stub handler `async () => ({ result: 'x' })` reproduces it, so it is the + // matcher's type, not this hook. Only this line is exempt. + // @ts-expect-error TS2589 in claude-code's Matcher> + on('tool.call', { tool: READY_TOOL_ID }, async ($): Promise<{ result: string }> => { const root = activeRoot const r = root === undefined ? undefined : await readRecord($, root) if (r === undefined || !r.on) return { result: `No auto-claude run is on; /${COMMAND} start first.` } @@ -731,7 +878,7 @@ export const register: Register = (on, options) => { on: true, phase, sessionId: await $.session.id(), - startedAt: Date.now(), + startedAt: now, beatAt: now, kicks: 0, kickTold: false, @@ -743,7 +890,7 @@ export const register: Register = (on, options) => { orphanTold = undefined await writeRecord($, next) if (phase === 'plan') { - const text = kickoffText(r.hint, `${r.remote}/${r.defaultBranch}`) + const text = kickoffText(r.hint, refOf(r), r.remote === LOCAL) $.clock.after(100, () => { $.prompt.submit({ text }).catch(() => $.ui.toast('auto-claude: the planning prompt was not submitted; send any message to start.')) }) diff --git a/auto-claude/hooks/ship.ts b/auto-claude/hooks/ship.ts index 3f3031f..ad4638d 100644 --- a/auto-claude/hooks/ship.ts +++ b/auto-claude/hooks/ship.ts @@ -1,23 +1,52 @@ import type { AutoRecord, ShipItem } from '../types' -// The loop's JSONL log as events, from the line after `offset` on. +// The loop's JSONL log as events, from after the last line already read. +// goal-mode keeps only its last 2000 lines (LOOP_LOG_MAX_LINES), so once the +// log is full its line count stops growing: a line offset alone then reads +// nothing new, ever. The cursor is the last line read (each carries a ts), +// looked for backwards from where it was; `offset` alone is the fallback for +// a record from before `last`. export type LogEntry = { event: string; goal?: string; branch?: string; worktree?: string; reason?: string } -export function readLog(text: string, offset: number): { entries: LogEntry[]; offset: number } { +export function readLog(text: string, offset: number, last?: string): { entries: LogEntry[]; offset: number; last?: string; held?: true } { const lines = text.split('\n').filter(line => line.trim() !== '') - // The loop caps its log; a shorter file than we read means it was trimmed. - const from = offset > lines.length ? Math.max(lines.length - 50, 0) : offset + let from: number + if (last !== undefined) { + // The line moves only towards the start when the log is trimmed. + const at = lines.lastIndexOf(last, Math.min(offset, lines.length) - 1) + // Not found in a log shorter than what was read: goal-mode rewrites the + // whole file on each append, so this is a read in the middle of that + // write. Nothing is read and the cursor stays; the next tick sees it whole. + // (A log cleared by hand reads the same way, until the caller decides it + // was replaced: see `held`.) + if (at < 0 && lines.length < offset) return { entries: [], offset, last, held: true } + // Not found otherwise: trimmed past it, so every line in the window is new. + from = at >= 0 ? at + 1 : 0 + } else { + // A shorter file than we read means it was trimmed. + from = offset > lines.length ? Math.max(lines.length - 50, 0) : offset + } const entries: LogEntry[] = [] - for (const line of lines.slice(from)) { + // The cursor only ever rests on a line that parsed: a torn last line is read + // again, whole, next time. + let cursor = from + let cursorLine = from > 0 ? lines[from - 1] : last + for (let i = from; i < lines.length; i++) { try { - const value = JSON.parse(line) as LogEntry + const value = JSON.parse(lines[i]!) as LogEntry if (typeof value.event === 'string') entries.push(value) + cursor = i + 1 + cursorLine = lines[i] } catch { - // A torn line is skipped. + // A torn last line waits for the next read; one in the middle is skipped. + if (i < lines.length - 1) { + cursor = i + 1 + cursorLine = lines[i] + } } } - return { entries, offset: lines.length } + return { entries, offset: cursor, last: cursorLine } } // goal-loop.gate as goal-mode reads it: starts still allowed, undefined for none. @@ -45,12 +74,14 @@ export function parseChecks(json: string): Checks { } // `gh pr view --json state,mergeable`: may it be merged now. -export function mergeable(json: string): { open: boolean; conflict: boolean; unknown: boolean } { +export function mergeable(json: string): { open: boolean; merged: boolean; conflict: boolean; unknown: boolean } { try { const v = JSON.parse(json) as { state?: string; mergeable?: string } - return { open: v.state === 'OPEN', conflict: v.mergeable === 'CONFLICTING', unknown: v.mergeable === 'UNKNOWN' } + // GitHub reports mergeable UNKNOWN for a merged PR, so merged is read first. + const merged = v.state === 'MERGED' + return { open: v.state === 'OPEN', merged, conflict: v.mergeable === 'CONFLICTING', unknown: !merged && v.mergeable === 'UNKNOWN' } } catch { - return { open: false, conflict: false, unknown: true } + return { open: false, merged: false, conflict: false, unknown: true } } } @@ -97,6 +128,11 @@ export type StallFacts = { maxNudges: number // Planning this long is no longer just planning (the first reminder's wait). planWaitMs: number + // How long nothing has moved (now - movedAt), when known. + idleMs?: number + // Ticks of the owner failed in a row, and the last error, once that is a streak. + tickFails?: number + tickError?: string } const ago = (now: number, at: number) => `${Math.max(0, Math.round((now - at) / 60_000))} min ago` @@ -107,6 +143,7 @@ export function diagnose(r: AutoRecord, f: StallFacts): string | undefined { if (r.sessionId !== undefined && r.sessionId !== f.sessionId && f.now - beat >= f.staleMs) { return `orphaned: the session running it (${r.sessionId.slice(0, 8)}) last ticked ${ago(f.now, beat)}, so nothing moves; /auto-claude resume here to go on` } + if (f.tickFails !== undefined) return `every tick fails (${f.tickFails} in a row), so nothing moves: ${(f.tickError ?? '').slice(0, 200)}` // phaseAt is when planning began or was last reminded: a reminder already // sent, or the first one's wait run out, means the plan is overdue. const since = r.phaseAt ?? r.startedAt @@ -125,7 +162,66 @@ export function diagnose(r: AutoRecord, f: StallFacts): string | undefined { const reply = (r.kickReply ?? 'no reply').replace(/\s+/g, ' ').slice(0, 200) return `a granted start is not taken: /goal-mode loop on ran ${r.kicks}× (last ${r.kickedAt !== undefined ? ago(f.now, r.kickedAt) : 'at an unknown time'}) and answered: ${reply}` } + const stuck = r.phase === 'run' ? ciStuck(r, f.now) : undefined + if (stuck !== undefined) { + return `PR #${stuck.pr ?? '?'} has waited ${stuck.minutes} min for CI to finish or GitHub to decide it can merge (a queued check with no runner?); gh pr checks ${stuck.pr ?? ''} shows which` + } + if (r.phase === 'run' && f.idleMs !== undefined && f.idleMs >= IDLE_NUDGE_MS && !isWaiting(r, f.now)) { + return `idle ${Math.round(f.idleMs / 60_000)} min: nothing was started, shipped or merged` + } return undefined } + +// "owner/name" of a GitHub remote URL (https or ssh form), else undefined. +// gh picks its target repository itself: in a fork that also has an `upstream` +// remote it can choose the parent, where the run's branch does not exist +// ("Head sha can't be blank ... Head ref must be a branch"; measured on a fork +// with no `gh repo set-default`). Every gh call names the pushed remote's repo. +export function repoSlug(url: string): string | undefined { + const m = url.trim().match(/github\.com[:/]([^/\s]+)\/([^/\s]+?)(?:\.git)?\/?$/) + return m === null ? undefined : `${m[1]}/${m[2]}` +} + +// Idle: the run is on and nothing has been started, shipped or merged for this +// long. The first nudge comes after IDLE_NUDGE_MS, the next ones IDLE_REPEAT_MS +// apart, MAX_IDLE_NUDGES in all; any progress starts the count again. +export const IDLE_NUDGE_MS = 15 * 60_000 +export const IDLE_REPEAT_MS = 30 * 60_000 +export const MAX_IDLE_NUDGES = 3 + +// What the run has done, so far: it changes when a goal starts or ends, a PR +// moves on or a milestone is merged or held. A repeating error note is not progress. +export function progressSig(r: Pick): string { + return [r.started, r.finished, r.merges, r.held, ...r.ships.map(s => `${s.headRef}:${s.state}:${s.pr ?? ''}`)].join('|') +} + +// A PR waits this long for its CI to finish (or for GitHub to decide whether it +// can be merged) before the wait itself counts as a stall: a check queued for +// a self-hosted runner that is offline stays pending for good. +export const CI_STUCK_MS = 90 * 60_000 + +// The in-flight PR waiting past CI_STUCK_MS, with how long, else undefined. +export function ciStuck(r: Pick, now: number): { pr?: number; minutes: number } | undefined { + const item = inFlight(r.ships) + if (item?.state !== 'ci' || item.note !== undefined || now - item.since < CI_STUCK_MS) return undefined + return { pr: item.pr, minutes: Math.round((now - item.since) / 60_000) } +} + +// Waiting on something that has its own clock is not idle: the loop building a +// goal, or a PR's CI pending with nothing wrong, for up to CI_STUCK_MS. +export function isWaiting(r: Pick, now: number): boolean { + if (r.started > r.finished) return true + const item = inFlight(r.ships) + return item?.state === 'ci' && item.note === undefined && ciStuck(r, now) === undefined +} + +// The number of the nudge due now (1..MAX_IDLE_NUDGES), or undefined. +export function idleDue(r: Pick, now: number): number | undefined { + if (!r.on || r.phase !== 'run' || r.movedAt === undefined) return undefined + const sent = r.idleNudges ?? 0 + if (sent >= MAX_IDLE_NUDGES) return undefined + if (sent === 0) return now - r.movedAt >= IDLE_NUDGE_MS ? 1 : undefined + return now - (r.idleNudgedAt ?? r.movedAt) >= IDLE_REPEAT_MS ? sent + 1 : undefined +} diff --git a/auto-claude/types/index.d.ts b/auto-claude/types/index.d.ts index 1cd9c0f..022909d 100644 --- a/auto-claude/types/index.d.ts +++ b/auto-claude/types/index.d.ts @@ -34,6 +34,10 @@ export type AutoRecord = { ciMode: 'github' | 'local' // Lines of .claude/goal-loop.log already read. logOffset: number + // The last line of it read: the cursor once goal-mode's 2000-line cap stops the count growing. + logLast?: string + // Ticks in a row the log read shorter than logOffset without logLast in it (a write in progress, or a cleared log). + logHeld?: number // Goals the loop has started (its `start` events). started: number // Goals the loop has finished (met, blocked or deferred): equal to started when it is idle. @@ -66,6 +70,12 @@ export type AutoRecord = { deploy?: { command: string; codeql: string[]; timeoutMs: number } deploys?: number note?: string + // Idle detection: what the run last did (ship.ts progressSig), when that last changed, + // and the idle nudges sent since (at most MAX_IDLE_NUDGES, reset by any progress). + sig?: string + movedAt?: number + idleNudges?: number + idleNudgedAt?: number } declare module 'claude-code' { diff --git a/scripts/ci.sh b/scripts/ci.sh index fdaa5d5..f4a2c26 100755 --- a/scripts/ci.sh +++ b/scripts/ci.sh @@ -2,8 +2,8 @@ # Created: 2026-10-27 # Purpose: The repository's CI, run the same way locally and in GitHub Actions: # `claude plugin validate` for the marketplace and every mod, `claude plugin -# test` for every mod, then `bun scripts/sync-shared.ts --check` for the -# copies generated from shared/. +# test` for every mod, tsc for each mod with a tsconfig.json, then +# `bun scripts/sync-shared.ts --check` for the copies generated from shared/. # Dependencies: claude (Claude Code CLI; no login needed), bun, perl # Test Status: run by .github/workflows/ci.yml and by the auto-claude milestone checks # @@ -49,6 +49,26 @@ for mod in "${mods[@]}"; do fi done +# Type check, for each mod with a tsconfig.json. Its base config lives in the +# generated .claude-plugin/types (gitignored, written by `claude plugin init`), +# so a checkout without it (GitHub Actions) says it skipped instead of failing. +# bun strips types without checking them, so `claude plugin test` alone never +# sees a type error. +for mod in "${mods[@]}"; do + [ -f "$mod/tsconfig.json" ] || continue + if [ ! -f "$mod/.claude-plugin/types/tsconfig.json" ]; then + echo "== $mod: tsc skipped (no generated .claude-plugin/types)" + continue + fi + echo "== $mod: tsc" + out=$(cd "$mod" && bun x --package typescript@5.6.3 tsc --noEmit -p . 2>&1) + errs=$(printf '%s\n' "$out" | grep -c 'error TS') + if [ "$errs" -ne 0 ]; then + printf '%s\n' "$out" | grep 'error TS' | head -20 + failed+=("$mod tsc ($errs errors)") + fi +done + echo "== shared: sync-shared --check" if ! out=$(bun scripts/sync-shared.ts --check 2>&1); then printf '%s\n' "$out"