diff --git a/plugins/studio-no-more-subscription/easyloop/.gitignore b/plugins/studio-no-more-subscription/easyloop/.gitignore new file mode 100644 index 00000000..6a9e5b5f --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/.gitignore @@ -0,0 +1,3 @@ +.data/ +*.swp +.DS_Store diff --git a/plugins/studio-no-more-subscription/easyloop/LICENSE b/plugins/studio-no-more-subscription/easyloop/LICENSE new file mode 100644 index 00000000..4b77ac7f --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/LICENSE @@ -0,0 +1,17 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + Copyright 2026 bro + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/plugins/studio-no-more-subscription/easyloop/README.md b/plugins/studio-no-more-subscription/easyloop/README.md new file mode 100644 index 00000000..2f0d0c71 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/README.md @@ -0,0 +1,101 @@ +# easyloop + +Self-orchestrating loop Skill for MiniMax Code. When the user invokes +**easyloop**, the model reads a fixed menu of loop patterns + quality +levels + optional plugins, picks what fits the task, and calls the +`finish_plugin_selection` MCP tool. The tool persists the selection +**along with each loop's one-line description** under a +`context_compacted_id`, then returns a short plan. From that point on, +the model carries only the id forward; the original menu can be +re-injected with `recall_selection("")`. + +Ships with three Skills: `easyloop`, `easyloop-prototype`, +`easyloop-release`. + +## Install + +```bash +cp -r easyloop ~/.mcode/plugins/local/ +mcode plugin enable easyloop@local +``` + +## Loop catalog + +13 loop patterns (12 standalone + 1 chain). Each ships with a one-line +description persisted alongside the selection: + +| Loop | One-line description | +| --- | --- | +| `explore` | Read and understand unfamiliar code; return a compressed summary. | +| `implement` | Write new code from a spec or plan. | +| `test` | Run tests, interpret failures, fix them until green. | +| `debug` | Diagnose a known bug, isolate root cause, patch it. | +| `refactor` | Improve structure or clarity without changing behavior. | +| `review` | Read code or a diff and surface concrete defects. | +| `document` | Write or update docs, docstrings, READMEs, comments. | +| `migrate` | Port code between frameworks, languages, or major versions. | +| `benchmark` | Profile and optimize performance, memory, or bundle size. | +| `security-audit` | Check for known vulnerability classes and fix them. | +| `deps-update` | Upgrade dependencies safely with tests + lockfile. | +| `release` | Version bump, changelog, tag, publish artifacts. | +| `chain` | Combine multiple loops in sequence (recommended for non-trivial work). | + +## Quality levels + +| Quality | Iteration cap | Verify (tests/lint/type) | When to use | +| --- | --- | --- | --- | +| `prototype` | 5 | off | Spike / exploration; throwaway code | +| `release` | 25 | on | Production / ship-ready output | + +## MCP tools + +| Tool | Purpose | +| --- | --- | +| `finish_plugin_selection` | Persist selection, return `context_compacted_id` + plan. | +| `recall_selection` | Re-inject the original menu + saved plan for an id. | +| `log_iteration` | Append an iteration audit entry. | +| `list_selections` | List all selections saved in this session. | + +## Sub-agent strategy + +The Skill body instructs the model to: + +- Use the `Task` tool (`task` / `delegate` / `spawn_agent`) with + `run_in_background: true` for independent sub-tasks. +- Spawn 3–5 background scouts at a time for multi-file exploration. +- Aggregate results before the next iteration. +- For implementation, one background worker implements while the main + thread prepares tests / lint. + +## Copyable example prompt + +> "easyloop: refactor `src/auth/` to use async/await end-to-end and ship." + +The model will: +1. Read the menu. +2. Call `finish_plugin_selection({ loops: ["explore","refactor","test","review"], quality: "release", plugins: ["mcode-think-filter"], notes: "auth async/await" })`. +3. Carry only the id forward. +4. Spawn background scouts in parallel. +5. Iterate refactor + test cycles up to the 25-turn cap. +6. `recall_selection("")` if it needs to look at the menu again. + +## Tests + +```bash +cd easyloop +node --test lib/*.test.mjs +``` + +Covers: +- Catalog has 12+ loops with descriptions. +- `renderMenu` covers every loop and quality. +- `validateSelection` rejects empty/unknown loops; accepts valid ones. +- `saveSelection` / `readSelection` round-trip with descriptions preserved. +- `readSelection` rejects malformed ids (path traversal guard). + +## Disclosure + +- Persists `selections/.json` and `iterations.jsonl` under + `${PLUGIN_DATA}`. +- Network: zero. +- Native binaries: none. diff --git a/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/hooks.json b/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/hooks.json new file mode 100644 index 00000000..025984c4 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/hooks.json @@ -0,0 +1,33 @@ +{ + "$schema": "https://minimax.io/schemas/mcode-hooks/0.1.0/hooks.schema.json", + "hooks": { + "SessionStart": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/log-loop.mjs", + "--data", + "${PLUGIN_DATA}", + "--phase", + "session-start" + ], + "timeout": 5000, + "once": false + } + ], + "Stop": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/log-loop.mjs", + "--data", + "${PLUGIN_DATA}", + "--phase", + "stop" + ], + "timeout": 5000, + "once": false + } + ] + } +} diff --git a/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/scripts/log-loop.mjs b/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/scripts/log-loop.mjs new file mode 100644 index 00000000..d33643d0 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/io.minimax.mcode/hooks/scripts/log-loop.mjs @@ -0,0 +1,40 @@ +#!/usr/bin/env node +// SessionStart / Stop observer for the easyloop plugin. Ensures +// ${PLUGIN_DATA}/selections/ exists and appends a small audit entry. + +import { argv, env } from 'node:process'; +import { mkdir, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; + +function parseArgs(args) { + const out = { data: null, phase: 'session-start' }; + for (let i = 0; i < args.length; i += 1) { + const a = args[i]; + if (a === '--data') { out.data = args[i + 1]; i += 1; } + else if (a === '--phase') { out.phase = args[i + 1]; i += 1; } + } + return out; +} + +function expandData(value) { + if (typeof value !== 'string') return null; + if (value.startsWith('${PLUGIN_DATA}')) { + const r = env.PLUGIN_DATA; + return r ? r + value.slice('${PLUGIN_DATA}'.length) : null; + } + return value; +} + +async function main() { + const args = parseArgs(argv.slice(2)); + const dataDir = expandData(args.data); + if (!dataDir) return; + await mkdir(join(dataDir, 'selections'), { recursive: true }); + await writeFile( + join(dataDir, 'session.log'), + JSON.stringify({ ts: new Date().toISOString(), phase: args.phase, pid: process.pid }) + '\n', + { flag: 'a' }, + ); +} + +main().catch(() => {}); diff --git a/plugins/studio-no-more-subscription/easyloop/lib/catalog.mjs b/plugins/studio-no-more-subscription/easyloop/lib/catalog.mjs new file mode 100644 index 00000000..aaf3f121 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/catalog.mjs @@ -0,0 +1,87 @@ +// catalog.mjs +// +// Static catalog of loops and quality levels. The plugin list passed to +// renderMenu() comes from lib/discover.mjs (CLI + filesystem scan). +// Pure data; no I/O. + +import { BUILTIN_OPTIONAL } from './discover.mjs'; + +export const LOOPS = [ + { name: 'explore', description: 'Read and understand unfamiliar code; return a compressed summary.' }, + { name: 'implement', description: 'Write new code from a spec or plan.' }, + { name: 'test', description: 'Run tests, interpret failures, fix them until green.' }, + { name: 'debug', description: 'Diagnose a known bug, isolate root cause, patch it.' }, + { name: 'refactor', description: 'Improve structure or clarity without changing behavior.' }, + { name: 'review', description: 'Read code or a diff and surface concrete defects.' }, + { name: 'document', description: 'Write or update docs, docstrings, READMEs, comments.' }, + { name: 'migrate', description: 'Port code between frameworks, languages, or major versions.' }, + { name: 'benchmark', description: 'Profile and optimize performance, memory, or bundle size.' }, + { name: 'security-audit',description: 'Check for known vulnerability classes and fix them.' }, + { name: 'deps-update', description: 'Upgrade dependencies safely with tests + lockfile.' }, + { name: 'release', description: 'Version bump, changelog, tag, publish artifacts.' }, + { name: 'chain', description: 'Combine multiple loops in sequence (recommended for non-trivial work).' }, +]; + +export const QUALITIES = [ + { name: 'prototype', description: 'Fast, throwaway, minimal tests, no docs.', iterationCap: 5, verify: false }, + { name: 'release', description: 'Full tests, type safety, docs, error handling.', iterationCap: 25, verify: true }, +]; + +// Backwards-compatible default (built-in optionals). Tests that don't +// care about discovery still work without changing their imports. +export const OPTIONAL_PLUGINS = BUILTIN_OPTIONAL.map((p) => ({ + name: p.name, + description: p.description, +})); + +// Render the selection menu as plain text the model reads in the Skill +// body. Pass `plugins` from lib/discover.mjs to include every installed +// plugin's one-line description; pass `null`/`undefined` to fall back +// to the built-in optionals; pass `[]` to explicitly show "none +// discovered". +export function renderMenu(plugins) { + // undefined/null → built-in fallback; [] → empty (caller knows there + // are no plugins). + const list = plugins === undefined || plugins === null + ? BUILTIN_OPTIONAL + : plugins; + const builtinNames = new Set(BUILTIN_OPTIONAL.map((p) => p.name)); + const lines = []; + lines.push('LOOP PATTERNS:'); + for (const l of LOOPS) { + lines.push(` ${l.name.padEnd(16)} — ${l.description}`); + } + lines.push(''); + lines.push('QUALITY LEVEL:'); + for (const q of QUALITIES) { + lines.push(` ${q.name.padEnd(16)} — ${q.description} (cap ${q.iterationCap} turns, verify=${q.verify})`); + } + lines.push(''); + lines.push('OPTIONAL PLUGINS (you may enable any of these for this task):'); + if (list.length === 0) { + lines.push(' (none discovered)'); + } else { + for (const p of list) { + const tag = builtinNames.has(p.name) ? ' (recommended)' : ''; + const desc = (p.description && p.description.length > 0) + ? p.description + : '(no description provided by manifest)'; + lines.push(` ${p.name.padEnd(28)} — ${desc}${tag}`); + } + } + lines.push(''); + lines.push('PARALLELISM:'); + lines.push(' Use the Task tool (`task`/`delegate`/`spawn_agent`) with `run_in_background: true` whenever'); + lines.push(' sub-tasks are independent. Maximize parallel calls in one turn. Spawn 3–5 background scouts'); + lines.push(' at a time for multi-file exploration, then aggregate.'); + return lines.join('\n'); +} + +// Look up loop metadata by name. +export function loopByName(name) { + return LOOPS.find((l) => l.name === name) || null; +} + +export function qualityByName(name) { + return QUALITIES.find((q) => q.name === name) || null; +} diff --git a/plugins/studio-no-more-subscription/easyloop/lib/catalog.test.mjs b/plugins/studio-no-more-subscription/easyloop/lib/catalog.test.mjs new file mode 100644 index 00000000..8ce3e06a --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/catalog.test.mjs @@ -0,0 +1,142 @@ +// Unit tests for catalog + planner. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { LOOPS, QUALITIES, renderMenu, loopByName, qualityByName } from './catalog.mjs'; +import { validateSelection, menuCharCount, setDiscoveredPlugins, getDiscoveredPlugins } from './planner.mjs'; +import { mergeCatalog, BUILTIN_OPTIONAL } from './discover.mjs'; + +test('catalog has 12+ loops with descriptions', () => { + assert.ok(LOOPS.length >= 12, `expected at least 12 loops, got ${LOOPS.length}`); + for (const l of LOOPS) { + assert.ok(l.name && typeof l.name === 'string'); + assert.ok(l.description && typeof l.description === 'string'); + } +}); + +test('renderMenu covers every loop and quality', () => { + const text = renderMenu(); + for (const l of LOOPS) assert.ok(text.includes(l.name), `menu missing ${l.name}`); + for (const q of QUALITIES) assert.ok(text.includes(q.name), `menu missing ${q.name}`); + assert.ok(text.includes('PARALLELISM')); +}); + +test('loopByName returns metadata or null', () => { + assert.ok(loopByName('explore').description.length > 0); + assert.equal(loopByName('does-not-exist'), null); +}); + +test('validateSelection rejects empty loops', () => { + const r = validateSelection({ loops: [], quality: 'release' }); + assert.equal(r.ok, false); + assert.match(r.error, /loops/); +}); + +test('validateSelection rejects unknown loop', () => { + const r = validateSelection({ loops: ['nope'], quality: 'release' }); + assert.equal(r.ok, false); + assert.match(r.error, /unknown loop/); +}); + +test('validateSelection accepts a valid selection and preserves descriptions', () => { + const r = validateSelection({ + loops: ['explore', 'implement', 'test'], + quality: 'release', + plugins: ['mcode-think-filter'], + notes: 'sample', + }); + assert.equal(r.ok, true); + assert.equal(r.selection.quality.name, 'release'); + assert.equal(r.selection.quality.iterationCap, 25); + assert.equal(r.selection.quality.verify, true); + assert.equal(r.selection.loops.length, 3); + for (const l of r.selection.loops) { + assert.ok(l.description && l.description.length > 0); + } + assert.equal(r.selection.plugins[0].name, 'mcode-think-filter'); + assert.equal(r.selection.notes, 'sample'); +}); + +test('validateSelection accepts unknown plugin with placeholder desc', () => { + const r = validateSelection({ + loops: ['explore'], + quality: 'prototype', + plugins: ['some-user-plugin'], + }); + assert.equal(r.ok, true); + assert.equal(r.selection.plugins[0].description, '(user plugin)'); +}); + +test('menuCharCount matches renderMenu length', () => { + assert.equal(menuCharCount(), renderMenu().length); +}); + +test('renderMenu with discovered plugins lists each one with its description', () => { + const plugins = [ + { name: 'mcode-computer-use', description: 'GUI automation.' }, + { name: 'my-custom-plugin', description: 'Custom tool that does X.' }, + ]; + const text = renderMenu(plugins); + assert.match(text, /mcode-computer-use/); + assert.match(text, /GUI automation/); + assert.match(text, /my-custom-plugin/); + assert.match(text, /Custom tool that does X\./); + // Recommended tag on the built-in + assert.match(text, /\(recommended\)/); +}); + +test('renderMenu with empty plugin list falls back gracefully', () => { + const text = renderMenu([]); + assert.match(text, /LOOP PATTERNS/); + assert.match(text, /QUALITY LEVEL/); + // No plugin section populated + assert.doesNotMatch(text, /\(recommended\)/); +}); + +test('mergeCatalog puts built-ins first and dedupes', () => { + const discovered = [ + { name: 'mcode-computer-use', description: 'overrides built-in description', marketplaceName: 'official' }, + { name: 'custom-plugin', description: 'a custom plugin', marketplaceName: 'local' }, + ]; + const merged = mergeCatalog(discovered); + assert.ok(merged.length >= 3, 'expected built-ins + custom'); + // First should be a built-in + assert.equal(merged[0].builtin, true); + // Built-in description should be replaced by richer discovered one + const cu = merged.find((p) => p.name === 'mcode-computer-use'); + assert.equal(cu.description, 'overrides built-in description'); + // Custom appears + assert.ok(merged.some((p) => p.name === 'custom-plugin')); +}); + +test('validateSelection stamps description from discovered plugins', () => { + setDiscoveredPlugins([ + { name: 'custom-x', description: 'does X', marketplaceName: 'local' }, + { name: 'mcode-computer-use', description: 'GUI automation', marketplaceName: 'official' }, + ]); + const r = validateSelection({ + loops: ['explore'], + quality: 'prototype', + plugins: ['custom-x', 'mcode-computer-use'], + }); + assert.equal(r.ok, true); + assert.equal(r.selection.plugins[0].description, 'does X'); + assert.equal(r.selection.plugins[1].description, 'GUI automation'); + setDiscoveredPlugins([]); +}); + +test('validateSelection falls back to (user plugin) for unknown names', () => { + setDiscoveredPlugins([]); + const r = validateSelection({ + loops: ['explore'], + quality: 'prototype', + plugins: ['never-seen-this'], + }); + assert.equal(r.ok, true); + assert.equal(r.selection.plugins[0].description, '(user plugin)'); +}); + +test('BUILTIN_OPTIONAL has the two suite plugins', () => { + const names = BUILTIN_OPTIONAL.map((p) => p.name); + assert.ok(names.includes('mcode-computer-use')); + assert.ok(names.includes('mcode-think-filter')); +}); diff --git a/plugins/studio-no-more-subscription/easyloop/lib/compact.mjs b/plugins/studio-no-more-subscription/easyloop/lib/compact.mjs new file mode 100644 index 00000000..e178b754 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/compact.mjs @@ -0,0 +1,67 @@ +// compact.mjs +// +// Selection persistence at ${PLUGIN_DATA}/selections/.json. Each +// selection stores: +// - the chosen loop names + their one-line descriptions +// - the chosen quality level + description + iteration cap + verify flag +// - the chosen plugin names + descriptions +// - the user's free-text notes +// - a timestamp and the original menu char count for audit +// +// recall_selection reads the same file back so the model can re-inject +// the original menu + the chosen items. + +import { mkdir, writeFile, rename, readFile, readdir } from 'node:fs/promises'; +import { join } from 'node:path'; +import { randomBytes } from 'node:crypto'; + +function shortId() { + return randomBytes(4).toString('hex'); // 8 hex chars; collisions negligible per session +} + +export async function saveSelection(pluginData, selection, menuChars) { + await mkdir(join(pluginData, 'selections'), { recursive: true }); + const id = `sel_${shortId()}`; + const record = { + context_compacted_id: id, + savedAt: new Date().toISOString(), + replacedChars: menuChars, + selection, + }; + const path = join(pluginData, 'selections', `${id}.json`); + const staged = path + '.staging'; + await writeFile(staged, JSON.stringify(record, null, 2), 'utf8'); + await rename(staged, path); + return record; +} + +export async function readSelection(pluginData, id) { + // Sanitize: only allow short hex ids. + if (!/^sel_[0-9a-f]{4,32}$/.test(id)) return null; + const path = join(pluginData, 'selections', `${id}.json`); + try { + const text = await readFile(path, 'utf8'); + return JSON.parse(text); + } catch { + return null; + } +} + +export async function listSelections(pluginData) { + const dir = join(pluginData, 'selections'); + try { + const names = await readdir(dir); + const out = []; + for (const name of names) { + if (!name.endsWith('.json')) continue; + try { + const text = await readFile(join(dir, name), 'utf8'); + const j = JSON.parse(text); + out.push({ context_compacted_id: j.context_compacted_id, savedAt: j.savedAt }); + } catch {} + } + return out.sort((a, b) => (b.savedAt || '').localeCompare(a.savedAt || '')); + } catch { + return []; + } +} diff --git a/plugins/studio-no-more-subscription/easyloop/lib/compact.test.mjs b/plugins/studio-no-more-subscription/easyloop/lib/compact.test.mjs new file mode 100644 index 00000000..c53eb017 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/compact.test.mjs @@ -0,0 +1,67 @@ +// Unit tests for compact.mjs using a temp dir. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { saveSelection, readSelection, listSelections } from './compact.mjs'; +import { renderMenu } from './catalog.mjs'; + +function tmpData() { + const dir = mkdtempSync(join(tmpdir(), 'easyloop-test-')); + return dir; +} + +test('saveSelection then readSelection round-trips with descriptions', async () => { + const dir = tmpData(); + try { + const selection = { + loops: [ + { name: 'explore', description: 'Read and understand unfamiliar code; return a compressed summary.' }, + { name: 'implement', description: 'Write new code from a spec or plan.' }, + ], + quality: { name: 'release', description: 'Full tests...', iterationCap: 25, verify: true }, + plugins: [{ name: 'mcode-think-filter', description: 'Use when context is critical.' }], + notes: 'n/a', + }; + const rec = await saveSelection(dir, selection, renderMenu().length); + assert.ok(rec.context_compacted_id.startsWith('sel_')); + assert.equal(rec.replacedChars, renderMenu().length); + + const back = await readSelection(dir, rec.context_compacted_id); + assert.ok(back); + assert.equal(back.context_compacted_id, rec.context_compacted_id); + assert.equal(back.selection.loops[0].name, 'explore'); + assert.match(back.selection.loops[0].description, /understand/); + assert.equal(back.selection.quality.name, 'release'); + assert.equal(back.selection.quality.iterationCap, 25); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('readSelection rejects malformed ids', async () => { + const dir = tmpData(); + try { + assert.equal(await readSelection(dir, 'nope'), null); + assert.equal(await readSelection(dir, '../../../etc/passwd'), null); + assert.equal(await readSelection(dir, ''), null); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('listSelections returns ids sorted newest first', async () => { + const dir = tmpData(); + try { + for (let i = 0; i < 3; i += 1) { + await saveSelection(dir, { loops: [{ name: 'explore', description: 'x' }], + quality: { name: 'release', description: 'y', iterationCap: 25, verify: true }, + plugins: [], notes: '' }, 1); + } + const list = await listSelections(dir); + assert.equal(list.length, 3); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/plugins/studio-no-more-subscription/easyloop/lib/discover.mjs b/plugins/studio-no-more-subscription/easyloop/lib/discover.mjs new file mode 100644 index 00000000..65c8fa8a --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/discover.mjs @@ -0,0 +1,183 @@ +// discover.mjs +// +// Discover installed plugins so the easyloop menu can present every +// plugin's description (one-liner) to the model. The model then picks +// which optional plugins to enable for the current task — including +// any custom plugins the user has installed locally. +// +// Discovery strategy (in order): +// 1. Run `mcode plugin list --json` if the CLI is on PATH. +// 2. Walk the local marketplace directory directly, reading each +// `plugin.json` (Agent Plugins 1.0 manifest). +// 3. Fall back to an empty list (the built-in OPTIONAL_PLUGINS still +// render in the menu). +// +// All operations are read-only and cached for the lifetime of the +// server process. + +import { spawnSync } from 'node:child_process'; +import { readFile, readdir, stat } from 'node:fs/promises'; +import { join, dirname } from 'node:path'; + +// Built-in known-good plugins. These always render in the menu even +// if discovery fails, so the model sees a useful default. +export const BUILTIN_OPTIONAL = [ + { name: 'mcode-computer-use', description: 'Use when GUI interaction is required (screenshot, click, scroll, keyboard).' }, + { name: 'mcode-think-filter', description: 'Use when context is critical (always on for release quality work).' }, +]; + +function runMcodeList() { + try { + const r = spawnSync('mcode', ['plugin', 'list', '--json'], { + encoding: 'utf8', + timeout: 5000, + }); + if (r.status !== 0 || !r.stdout) return null; + return JSON.parse(r.stdout); + } catch { + return null; + } +} + +function runMcodeMarketplaceList() { + try { + const r = spawnSync('mcode', ['plugin', 'marketplace', 'list', '--json'], { + encoding: 'utf8', + timeout: 5000, + }); + if (r.status !== 0 || !r.stdout) return null; + const arr = JSON.parse(r.stdout); + return Array.isArray(arr) ? arr : []; + } catch { + return null; + } +} + +async function readManifest(dir) { + try { + const text = await readFile(join(dir, 'plugin.json'), 'utf8'); + const j = JSON.parse(text); + if (j && typeof j.name === 'string') { + return { + name: j.name, + description: typeof j.description === 'string' ? j.description : '', + marketplaceName: 'local', + }; + } + } catch {} + return null; +} + +async function scanLocalDir(dir) { + const out = []; + let entries; + try { + entries = await readdir(dir); + } catch { + return out; + } + for (const entry of entries) { + const p = join(dir, entry); + try { + const st = await stat(p); + if (!st.isDirectory()) continue; + } catch { + continue; + } + // Two layouts: dir//plugin.json, or dir/plugin.json directly. + const manifest = await readManifest(p) || + await readManifest(join(p, entries[0] === 'plugin.json' ? '' : '')); + // First try p/plugin.json, then p//plugin.json. + const direct = await readManifest(p); + if (direct) { + out.push(direct); + continue; + } + // Subdirectory layout: dir///plugin.json — collect all. + let subEntries; + try { + subEntries = await readdir(p); + } catch { + continue; + } + for (const sub of subEntries) { + const sp = join(p, sub); + try { + const sst = await stat(sp); + if (!sst.isDirectory()) continue; + } catch { + continue; + } + const m = await readManifest(sp); + if (m) out.push(m); + } + } + return out; +} + +export async function discoverPlugins({ env = process.env } = {}) { + // Prefer the mcode CLI when available — it knows about official plugins too. + const fromCli = runMcodeList(); + if (fromCli && Array.isArray(fromCli.installed)) { + return fromCli.installed + .filter((p) => p && typeof p.name === 'string') + .map((p) => ({ + name: p.name, + description: typeof p.description === 'string' ? p.description : '', + marketplaceName: typeof p.marketplaceName === 'string' ? p.marketplaceName : 'unknown', + enabled: p.enabled !== false, + })); + } + + // Fallback: scan local marketplace directory. + const mkt = runMcodeMarketplaceList(); + if (Array.isArray(mkt)) { + const local = mkt.find((m) => m && m.kind === 'directory' && typeof m.path === 'string'); + if (local) { + const found = await scanLocalDir(local.path); + if (found.length > 0) return found; + } + } + + // Last-ditch: try PLUGIN_DATA's parent, since PLUGIN_DATA typically + // sits under the local marketplace. + if (env.PLUGIN_DATA) { + const guess = dirname(env.PLUGIN_DATA); + const found = await scanLocalDir(guess); + if (found.length > 0) return found; + } + + return []; +} + +// Combine built-in optionals with discovered plugins. De-duplicate by +// name. The discovered plugin's description (from its plugin.json +// manifest) takes precedence when present — the manifest is the +// plugin's authoritative self-description. Falls back to the built-in +// description when discovery didn't provide one (or didn't run). +export function mergeCatalog(discovered) { + const byName = new Map(); + for (const b of BUILTIN_OPTIONAL) byName.set(b.name, { ...b, builtin: true }); + for (const d of discovered) { + if (!d || !d.name) continue; + const existing = byName.get(d.name); + if (existing) { + // Discovered description wins if non-empty; otherwise keep built-in. + if (d.description && d.description.length > 0) { + existing.description = d.description; + } + existing.marketplaceName = d.marketplaceName || existing.marketplaceName; + // If discovered says disabled, mark it. + if (d.enabled === false) existing.enabled = false; + } else { + byName.set(d.name, { ...d, builtin: false }); + } + } + const out = Array.from(byName.values()); + out.sort((a, b) => { + // Built-in first, then alphabetical. + if (a.builtin !== b.builtin) return a.builtin ? -1 : 1; + return a.name.localeCompare(b.name); + }); + return out; +} diff --git a/plugins/studio-no-more-subscription/easyloop/lib/discover.test.mjs b/plugins/studio-no-more-subscription/easyloop/lib/discover.test.mjs new file mode 100644 index 00000000..3b3a6764 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/discover.test.mjs @@ -0,0 +1,46 @@ +// Unit tests for lib/discover.mjs using a fake local marketplace. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { mergeCatalog, BUILTIN_OPTIONAL } from './discover.mjs'; + +test('mergeCatalog returns built-ins when discovered list is empty', () => { + const merged = mergeCatalog([]); + assert.equal(merged.length, BUILTIN_OPTIONAL.length); + for (const p of merged) assert.equal(p.builtin, true); +}); + +test('mergeCatalog preserves order (builtins first, then alphabetical)', () => { + const merged = mergeCatalog([ + { name: 'zeta-plugin', description: 'z', marketplaceName: 'local' }, + { name: 'alpha-plugin', description: 'a', marketplaceName: 'local' }, + ]); + // First two are builtins, then alpha, then zeta. + assert.equal(merged[0].builtin, true); + assert.equal(merged[1].builtin, true); + const customs = merged.filter((p) => !p.builtin); + assert.equal(customs[0].name, 'alpha-plugin'); + assert.equal(customs[1].name, 'zeta-plugin'); +}); + +test('mergeCatalog overrides built-in description when richer one is discovered', () => { + const merged = mergeCatalog([ + { name: 'mcode-computer-use', description: 'richer desc from discovery', marketplaceName: 'official' }, + ]); + const cu = merged.find((p) => p.name === 'mcode-computer-use'); + assert.equal(cu.description, 'richer desc from discovery'); +}); + +test('mergeCatalog ignores malformed discovered entries', () => { + const merged = mergeCatalog([ + null, + { name: '', description: 'empty name' }, + { description: 'no name' }, + { name: 'ok-plugin', description: 'fine', marketplaceName: 'local' }, + ]); + const customs = merged.filter((p) => !p.builtin); + assert.equal(customs.length, 1); + assert.equal(customs[0].name, 'ok-plugin'); +}); diff --git a/plugins/studio-no-more-subscription/easyloop/lib/planner.mjs b/plugins/studio-no-more-subscription/easyloop/lib/planner.mjs new file mode 100644 index 00000000..76884669 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/lib/planner.mjs @@ -0,0 +1,83 @@ +// planner.mjs +// +// Validate the selection JSON the model passes to finish_plugin_selection +// against the catalog. Pure. + +import { renderMenu, loopByName, qualityByName, OPTIONAL_PLUGINS } from './catalog.mjs'; + +// Optional: a list of discovered plugins (with descriptions) so we can +// stamp the model-chosen plugin with its real description rather than +// a "(user plugin)" placeholder. +let _discoveredPlugins = null; +export function setDiscoveredPlugins(plugins) { + _discoveredPlugins = Array.isArray(plugins) ? plugins : []; +} +export function getDiscoveredPlugins() { + return _discoveredPlugins || []; +} + +export function validateSelection(sel) { + if (!sel || typeof sel !== 'object') { + return { ok: false, error: 'selection must be an object' }; + } + const errors = []; + + // loops: array of loop names + const loops = Array.isArray(sel.loops) ? sel.loops : []; + if (loops.length === 0) errors.push('loops must be a non-empty array'); + const loopDescs = []; + for (const name of loops) { + if (typeof name !== 'string') { errors.push(`loop entry must be a string, got ${typeof name}`); continue; } + const meta = loopByName(name); + if (!meta) errors.push(`unknown loop: ${name}`); + else loopDescs.push({ name: meta.name, description: meta.description }); + } + + // quality: must match a known level + let qualityDesc = null; + if (typeof sel.quality !== 'string') errors.push('quality must be a string'); + else { + const q = qualityByName(sel.quality); + if (!q) errors.push(`unknown quality: ${sel.quality}`); + else qualityDesc = { name: q.name, description: q.description, iterationCap: q.iterationCap, verify: q.verify }; + } + + // plugins: array of plugin names — accept ANY string. Look up the + // description from discovered plugins first, then OPTIONAL_PLUGINS, + // then fall back to a generic label. + const plugins = Array.isArray(sel.plugins) ? sel.plugins : []; + const pluginDescs = []; + const known = new Map(); + for (const p of _discoveredPlugins || []) if (p && p.name) known.set(p.name, p); + for (const p of OPTIONAL_PLUGINS) if (!known.has(p.name)) known.set(p.name, p); + for (const name of plugins) { + if (typeof name !== 'string') { errors.push(`plugin entry must be a string`); continue; } + const meta = known.get(name); + if (meta && meta.description) { + pluginDescs.push({ name: meta.name, description: meta.description }); + } else if (meta) { + pluginDescs.push({ name: meta.name, description: '(no description)' }); + } else { + pluginDescs.push({ name, description: '(user plugin)' }); + } + } + + // notes: optional free text + const notes = typeof sel.notes === 'string' ? sel.notes : ''; + + if (errors.length > 0) return { ok: false, error: errors.join('; ') }; + + return { + ok: true, + selection: { + loops: loopDescs, + quality: qualityDesc, + plugins: pluginDescs, + notes, + }, + }; +} + +export function menuCharCount() { + return renderMenu(_discoveredPlugins).length; +} diff --git a/plugins/studio-no-more-subscription/easyloop/mcp.json b/plugins/studio-no-more-subscription/easyloop/mcp.json new file mode 100644 index 00000000..f67b5c4a --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/mcp.json @@ -0,0 +1,10 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json", + "mcpServers": { + "easyloop": { + "type": "stdio", + "command": "node", + "args": ["./server/easyloop.mjs"] + } + } +} diff --git a/plugins/studio-no-more-subscription/easyloop/plugin.json b/plugins/studio-no-more-subscription/easyloop/plugin.json new file mode 100644 index 00000000..67019658 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/plugin.json @@ -0,0 +1,11 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "easyloop", + "version": "0.1.0", + "description": "Self-orchestrating loop Skill. When invoked, the LLM picks a loop pattern, quality level, and optional plugins, then calls finish_plugin_selection to compact the selection menu out of context under a context_compacted_id. Ships with three Skills: easyloop, easyloop-prototype, easyloop-release.", + "author": { + "name": "bro" + }, + "license": "Apache-2.0", + "keywords": ["mcode", "loop", "orchestration", "planner", "subagent"] +} diff --git a/plugins/studio-no-more-subscription/easyloop/server/easyloop.mjs b/plugins/studio-no-more-subscription/easyloop/server/easyloop.mjs new file mode 100644 index 00000000..90aeb3ba --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/server/easyloop.mjs @@ -0,0 +1,273 @@ +// server/easyloop.mjs +// +// stdio MCP server for the easyloop plugin. Tools: +// - finish_plugin_selection +// - recall_selection +// - log_iteration +// - list_selections + +import { createInterface } from 'node:readline'; +import { fileURLToPath } from 'node:url'; +import { dirname, join } from 'node:path'; +import { mkdir } from 'node:fs/promises'; + +import { renderMenu, LOOPS, QUALITIES } from '../lib/catalog.mjs'; +import { validateSelection, setDiscoveredPlugins } from '../lib/planner.mjs'; +import { saveSelection, readSelection, listSelections } from '../lib/compact.mjs'; +import { discoverPlugins, mergeCatalog } from '../lib/discover.mjs'; + +const PROTOCOL_VERSION = '2025-06-18'; +const SERVER_INFO = { name: 'easyloop', version: '0.1.0' }; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const PLUGIN_ROOT = process.env.PLUGIN_ROOT || join(HERE, '..'); +const PLUGIN_DATA = process.env.PLUGIN_DATA || join(PLUGIN_ROOT, '.data'); +await mkdir(PLUGIN_DATA, { recursive: true }); +await mkdir(join(PLUGIN_DATA, 'selections'), { recursive: true }); + +// Discover every installed plugin at session start. The model sees +// each plugin's `description` from its `plugin.json` manifest as a +// one-liner in the menu, and can enable any of them — including custom +// plugins the user has installed locally — for the current task. +const discovered = await discoverPlugins(); +const catalog = mergeCatalog(discovered); +setDiscoveredPlugins(catalog); + +const MENU_TEXT = renderMenu(catalog); +const MENU_CHARS = MENU_TEXT.length; + +function textContent(s) { return { type: 'text', text: s }; } + +const TOOLS = [ + { + name: 'get_menu', + description: + 'Return the full easyloop selection menu as text. The menu lists every loop, ' + + 'quality level, and the descriptions of every installed plugin (built-in ' + + 'and custom). Call this FIRST when the easyloop Skill activates, before ' + + 'calling finish_plugin_selection.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, + { + name: 'list_available_plugins', + description: + 'List every plugin installed in the local marketplace with its one-line ' + + 'description (from each plugin\'s plugin.json manifest). Use this to see ' + + 'which custom plugins are available before finish_plugin_selection.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, + { + name: 'finish_plugin_selection', + description: + 'Persist the model\'s loop/quality/plugin selection and return a ' + + 'context_compacted_id. The Skill body instructs the model to drop ' + + 'the original menu text from its next turn\'s output and carry only ' + + 'the id forward. Use recall_selection to re-inject the menu later.', + inputSchema: { + type: 'object', + properties: { + loops: { type: 'array', items: { type: 'string' } }, + quality: { type: 'string', enum: ['prototype', 'release'] }, + plugins: { type: 'array', items: { type: 'string' } }, + notes: { type: 'string' }, + }, + required: ['loops', 'quality'], + additionalProperties: false, + }, + }, + { + name: 'recall_selection', + description: 'Re-inject a previously compacted selection menu by id.', + inputSchema: { + type: 'object', + properties: { context_compacted_id: { type: 'string' } }, + required: ['context_compacted_id'], + additionalProperties: false, + }, + }, + { + name: 'log_iteration', + description: 'Append an iteration audit entry to the session log.', + inputSchema: { + type: 'object', + properties: { + turn: { type: 'integer' }, + loop: { type: 'string' }, + summary: { type: 'string' }, + }, + required: ['turn', 'loop', 'summary'], + additionalProperties: false, + }, + }, + { + name: 'list_selections', + description: 'List all selections saved in this session.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, +]; + +async function toolFinishPluginSelection(args) { + const v = validateSelection(args); + if (!v.ok) { + return { + content: [textContent(`finish_plugin_selection: ${v.error}`)], + isError: true, + structuredContent: { error: v.error }, + }; + } + const record = await saveSelection(PLUGIN_DATA, v.selection, MENU_CHARS); + + const planText = renderPlan(v.selection); + return { + content: [textContent( + `selection saved as ${record.context_compacted_id}.\n\n` + + `PLAN:\n${planText}\n\n` + + `The original menu (${MENU_CHARS} chars) is now compacted; carry only this id ` + + `forward. Use recall_selection("${record.context_compacted_id}") to re-inject the menu later.`, + )], + structuredContent: { + context_compacted_id: record.context_compacted_id, + replaced_chars: MENU_CHARS, + plan_text: planText, + selection: v.selection, + }, + }; +} + +function renderPlan(s) { + const lines = []; + lines.push(`Loops: ${s.loops.map((l) => l.name).join(', ')}`); + for (const l of s.loops) lines.push(` - ${l.name}: ${l.description}`); + lines.push(`Quality: ${s.quality.name} (cap ${s.quality.iterationCap} turns, verify=${s.quality.verify})`); + lines.push(` - ${s.quality.description}`); + if (s.plugins.length > 0) { + lines.push(`Plugins:`); + for (const p of s.plugins) lines.push(` - ${p.name}: ${p.description}`); + } else { + lines.push(`Plugins: none`); + } + if (s.notes) lines.push(`Notes: ${s.notes}`); + return lines.join('\n'); +} + +async function toolRecallSelection(args) { + const id = args.context_compacted_id; + const record = await readSelection(PLUGIN_DATA, id); + if (!record) { + return { + content: [textContent(`recall_selection: not found: ${id}`)], + isError: true, + structuredContent: { found: false }, + }; + } + const planText = renderPlan(record.selection); + return { + content: [textContent( + `MENU (original, ${record.replacedChars} chars):\n${MENU_TEXT}\n\n` + + `SAVED PLAN:\n${planText}\n`, + )], + structuredContent: { + context_compacted_id: record.context_compacted_id, + replaced_chars: record.replacedChars, + plan_text: planText, + selection: record.selection, + }, + }; +} + +async function toolLogIteration(args) { + const path = join(PLUGIN_DATA, 'iterations.jsonl'); + const { writeFile } = await import('node:fs/promises'); + const line = JSON.stringify({ + ts: new Date().toISOString(), + turn: args.turn, + loop: args.loop, + summary: args.summary, + }) + '\n'; + await writeFile(path, line, { flag: 'a' }); + return { + content: [textContent(`logged iteration ${args.turn} (${args.loop})`)], + structuredContent: { logged: true }, + }; +} + +async function toolGetMenu() { + return { + content: [textContent(MENU_TEXT)], + structuredContent: { + menu_chars: MENU_CHARS, + plugins: catalog, + loops: LOOPS.map(({ name, description }) => ({ name, description })), + qualities: QUALITIES, + }, + }; +} + +async function toolListAvailablePlugins() { + return { + content: [textContent( + catalog.map((p) => `${p.name}: ${p.description}`).join('\n') || '(none discovered)', + )], + structuredContent: { plugins: catalog }, + }; +} + +async function toolListSelections() { + const items = await listSelections(PLUGIN_DATA); + return { + content: [textContent(JSON.stringify(items, null, 2))], + structuredContent: { selections: items }, + }; +} + +const ROUTES = { + get_menu: toolGetMenu, + list_available_plugins: toolListAvailablePlugins, + finish_plugin_selection: toolFinishPluginSelection, + recall_selection: toolRecallSelection, + log_iteration: toolLogIteration, + list_selections: toolListSelections, +}; + +async function handle(message) { + const { method, params, id } = message; + if (method === 'initialize') { + return { + result: { + protocolVersion: params?.protocolVersion ?? PROTOCOL_VERSION, + capabilities: { tools: {} }, + serverInfo: SERVER_INFO, + }, + }; + } + if (method === 'notifications/initialized') return { result: {} }; + if (method === 'tools/list') return { result: { tools: TOOLS } }; + if (method === 'tools/call') { + const name = params?.name; + const fn = ROUTES[name]; + if (!fn) return { error: { code: -32601, message: `unknown tool: ${name}` } }; + try { + const out = await fn(params?.arguments || {}); + return { result: out }; + } catch (e) { + return { + result: { + content: [textContent(`error: ${e.message}`)], + isError: true, + structuredContent: { error: { code: 'INTERNAL', message: e.message } }, + }, + }; + } + } + return { error: { code: -32601, message: `unknown method: ${method}` } }; +} + +const rl = createInterface({ input: process.stdin, crlfDelay: Infinity }); +rl.on('line', async (line) => { + if (!line.trim()) return; + let msg; + try { msg = JSON.parse(line); } catch { return; } + if (msg.id === undefined) return; + const resp = await handle(msg); + process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', id: msg.id, ...resp })}\n`); +}); diff --git a/plugins/studio-no-more-subscription/easyloop/skills/easyloop-prototype/SKILL.md b/plugins/studio-no-more-subscription/easyloop/skills/easyloop-prototype/SKILL.md new file mode 100644 index 00000000..8be82858 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/skills/easyloop-prototype/SKILL.md @@ -0,0 +1,60 @@ +--- +name: easyloop-prototype +description: EasyLoop variant for prototype-quality work. Defaults to quality=prototype (5-turn cap, no tests, no docs). Spawns background Task agents in parallel but caps each loop at fewer iterations. +license: Apache-2.0 +compatibility: Requires easyloop plugin. +metadata: + author: bro + version: "0.1.0" +--- + +# EasyLoop — Prototype variant + +Same selection menu as `easyloop`, but the **defaults** are tuned for +spike/exploration quality: + +- Iteration cap: **5 turns** (vs 25 for release). +- Verify (tests + linter + type-check): **off**. +- Sub-agent count: **2–3 background Tasks** (not 3–5). +- Doc generation: skip. + +## Step 1: call `get_menu` to see the live plugin list + +Before calling `finish_plugin_selection`, call `get_menu`. It lists +every installed plugin (built-in + custom) with its one-line +description. Pick the plugins whose descriptions match the task. + +## Recommended selection for most prototype tasks + +``` +finish_plugin_selection({ + loops: ["explore", "implement"], + quality: "prototype", + plugins: [], // or any custom plugin that fits + notes: "" +}) +``` + +## Call shape (same as easyloop) + +``` +finish_plugin_selection({ + loops: [...], // array of loop names from get_menu + quality: "prototype", // hard-coded + plugins: [...], // optional, from get_menu + notes: "..." +}) +``` + +## After the call + +- Move fast. Use 1–3 iterations per loop. Don't run tests; smoke-check + by reading the diff. +- If you discover the prototype is going to ship, **re-invoke the + `easyloop-release` Skill** to switch to release quality — call + `finish_plugin_selection` again with `quality: "release"`. + +## Disclosure + +Same as `easyloop`: persists to `${PLUGIN_DATA}`, no network, no native +binaries. diff --git a/plugins/studio-no-more-subscription/easyloop/skills/easyloop-release/SKILL.md b/plugins/studio-no-more-subscription/easyloop/skills/easyloop-release/SKILL.md new file mode 100644 index 00000000..1fcfc413 --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/skills/easyloop-release/SKILL.md @@ -0,0 +1,68 @@ +--- +name: easyloop-release +description: EasyLoop variant for release-quality work. Defaults to quality=release (25-turn cap, full tests + linter + type-check, docstrings). Spawns 3–5 background Task agents per exploration round and aggregates before implementing. +license: Apache-2.0 +compatibility: Requires easyloop plugin. +metadata: + author: bro + version: "0.1.0" +--- + +# EasyLoop — Release variant + +Same selection menu as `easyloop`, but the **defaults** are tuned for +ship-ready output: + +- Iteration cap: **25 turns**. +- Verify: **on** — run tests, linter, type-check after every + implement/test/refactor cycle. +- Sub-agent count: **3–5 background Tasks** per exploration round. +- Doc generation: docstring stubs at minimum. + +## Step 1: call `get_menu` to see the live plugin list + +Before calling `finish_plugin_selection`, call `get_menu`. It lists +every installed plugin (built-in + custom) with its one-line +description. Release-quality work should enable: + +- `mcode-think-filter` (recommended for release quality — long + iterations benefit from lean transcripts). +- Any custom plugin whose description matches the task. + +## Recommended selection for most release tasks + +``` +finish_plugin_selection({ + loops: ["explore", "implement", "test", "review", "document"], + quality: "release", + plugins: ["mcode-think-filter", /* + any relevant custom plugin */], + notes: "" +}) +``` + +## Call shape (same as easyloop) + +``` +finish_plugin_selection({ + loops: [...], + quality: "release", + plugins: [...], // from get_menu + notes: "..." +}) +``` + +## After the call + +- Always run the project's test suite at the end of `implement` and + every `test` cycle. Fix until green before declaring the loop done. +- For migration / dependency-update work, run `test` twice (once + before changes as a baseline, once after). +- For `release` loop, bump the version in the project's version file + and update CHANGELOG.md. +- Use `log_iteration({ turn, loop, summary })` to audit which loops + ran and how many turns each consumed. + +## Disclosure + +Same as `easyloop`: persists to `${PLUGIN_DATA}`, no network, no native +binaries. diff --git a/plugins/studio-no-more-subscription/easyloop/skills/easyloop/SKILL.md b/plugins/studio-no-more-subscription/easyloop/skills/easyloop/SKILL.md new file mode 100644 index 00000000..ebb5283d --- /dev/null +++ b/plugins/studio-no-more-subscription/easyloop/skills/easyloop/SKILL.md @@ -0,0 +1,108 @@ +--- +name: easyloop +description: Self-orchestrating loop Skill. When the user invokes easyloop, the LLM inspects the task, picks a loop pattern (explore/implement/test/debug/refactor/review/document/migrate/benchmark/security-audit/deps-update/release/chain), picks a quality level (prototype/release), and optionally enables extra plugins. The selection menu is then compacted out of context via the finish_plugin_selection MCP tool and stored under a context_compacted_id. The model proceeds with maximum parallelism via the Task tool. +license: Apache-2.0 +compatibility: Requires easyloop plugin; benefits from the Task tool (`task`/`delegate`/`spawn_agent`) being available. +metadata: + author: bro + version: "0.1.0" +--- + +# EasyLoop — Self-Orchestrating Loop + +When this Skill activates, you must: + +1. **Call `get_menu`** first. It returns the live selection menu — every + loop, quality level, AND every installed plugin (built-in + custom) + with each plugin's one-line description from its `plugin.json` + manifest. This is how you discover which custom plugins are + available; this Skill body alone can't list them. +2. **Decide** which loops fit the task, which quality level, and which + optional plugins to enable. Read each plugin's one-line description + carefully — that's the only signal you have. +3. **Call `finish_plugin_selection`** with your choices as JSON. The + tool persists your selection (including the chosen loops' AND + plugins' descriptions) under a `context_compacted_id` and returns + the short plan. +4. **Drop the menu text from your next turn's output.** Carry only the + `context_compacted_id` and the returned plan. The original menu is + re-injectable via `recall_selection("")` if you need to look + back. + +## Loop catalog (static — always available) + +| Loop | Description | +| --- | --- | +| `explore` | Read and understand unfamiliar code; return a compressed summary. | +| `implement` | Write new code from a spec or plan. | +| `test` | Run tests, interpret failures, fix them until green. | +| `debug` | Diagnose a known bug, isolate root cause, patch it. | +| `refactor` | Improve structure or clarity without changing behavior. | +| `review` | Read code or a diff and surface concrete defects. | +| `document` | Write or update docs, docstrings, READMEs, comments. | +| `migrate` | Port code between frameworks, languages, or major versions. | +| `benchmark` | Profile and optimize performance, memory, or bundle size. | +| `security-audit` | Check for known vulnerability classes and fix them. | +| `deps-update` | Upgrade dependencies safely with tests + lockfile. | +| `release` | Version bump, changelog, tag, publish artifacts. | +| `chain` | Combine multiple loops in sequence (recommended for non-trivial work). | + +## Quality levels (static — always available) + +| Quality | Iteration cap | Verify (tests/lint/type) | When to use | +| --- | --- | --- | --- | +| `prototype` | 5 | off | Spike / exploration; throwaway code | +| `release` | 25 | on | Production / ship-ready output | + +## Optional plugins (DYNAMIC — call `get_menu` to see what's installed) + +`get_menu` lists every plugin in the local marketplace with its +one-line description. The model picks which to enable for the current +task based on those descriptions. The two well-known ones from this +suite are auto-marked `(recommended)`: + +- `mcode-computer-use` — GUI automation (screenshot, click, scroll, keyboard). +- `mcode-think-filter` — keeps transcripts lean. + +Custom plugins (anything the user has dropped into the local marketplace) +are listed too, with their `description` field from `plugin.json`. If +you see a plugin whose description matches the task, you may select it. + +## Call shape + +``` +finish_plugin_selection({ + loops: ["explore", "implement", "test"], // array of loop names + quality: "release", // "prototype" or "release" + plugins: ["mcode-think-filter"], // optional; can be empty or include custom plugins + notes: "short human-readable plan" // optional +}) +``` + +The tool returns `{ context_compacted_id, replaced_chars, plan_text, selection }`. +`context_compacted_id` is the only thing you carry forward. + +## After the call + +- Run the chosen loops in sequence (use `chain` if multiple). +- Stay under `quality.iterationCap` turns. +- If `quality.verify` is true, run the project's tests + linter + type-check at + the end of every implement/test/refactor cycle. +- For multi-file work, **spawn 3–5 background Task agents in parallel** + per exploration round. Aggregate their results, then plan. +- If you need to look back at the menu, call + `recall_selection("")`. + +## Parallelism + +Use the Task tool (`task`/`delegate`/`spawn_agent`) with +`run_in_background: true` whenever sub-tasks are independent. Spawn +3–5 background scouts at a time for multi-file exploration, then +aggregate. + +## Disclosure + +- Persists `selections/.json` and `iterations.jsonl` under + `${PLUGIN_DATA}`. +- Network: zero. +- Native binaries: none. diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/.gitignore b/plugins/studio-no-more-subscription/mcode-computer-use/.gitignore new file mode 100644 index 00000000..6a9e5b5f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/.gitignore @@ -0,0 +1,3 @@ +.data/ +*.swp +.DS_Store diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/LICENSE b/plugins/studio-no-more-subscription/mcode-computer-use/LICENSE new file mode 100644 index 00000000..4b77ac7f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/LICENSE @@ -0,0 +1,17 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + Copyright 2026 bro + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/README.md b/plugins/studio-no-more-subscription/mcode-computer-use/README.md new file mode 100644 index 00000000..e98614be --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/README.md @@ -0,0 +1,229 @@ +# mcode-computer-use + +Drive the user's desktop through a **noise-grid addressing protocol**: +the model sees a screenshot with two deterministic random-noise grids +overlaid, and issues clicks, scrolls, drags, and keyboard input via +`(color1, color2, pos, action)` tuples instead of raw pixel +coordinates. + +This plugin bundles: + +- A **trajectory summarizer** that fires every N turns (default 10) + and lets the model replace older frames with text descriptions while + keeping the originals on disk. +- A **private headless browser** backend (Chrome via CDP, Firefox via + Marionette) that runs under the agent's own permissions — no + OS-level screen-capture permission required. +- An **opt-in Chromium profile inheritance** flow that copies the + user's Chrome profile (cookies, cache, login state) into a temp dir + so the agent can drive a browser with the user's existing sessions. +- An **Agent Activity Dashboard** served on a random localhost port + with SSE-driven live updates and inline permission UI. + +## Install + +```bash +cp -r mcode-computer-use ~/.mcode/plugins/local/ +mcode plugin enable mcode-computer-use@local +``` + +## Three run modes + +| Mode | Permission | Login state | Best for | +| --- | --- | --- | --- | +| OS desktop (`xdotool` etc.) | OS permission required | User's live session | Full desktop automation | +| Private Chrome (CDP) | None | Fresh, no cookies | Web automation without OS access | +| Inherited Chrome profile | None (after one-time consent) | User's existing session | Authenticated web tasks | + +## Browser backends + +| Backend | Permission | Login state | Caveat | +| --- | --- | --- | --- | +| Private Chrome (CDP) | None | Fresh | Works on any system with Chrome | +| Inherited Chrome profile | None (after consent) | User's existing session | Copies profile to temp; source untouched | +| Private Firefox (Marionette) | None | Fresh | Firefox 155+ has a Marionette bug — see below | + +### Firefox 155 caveat + +Firefox 155's headless mode ships with a Marionette TCP server that +accepts connections but **does not respond to WebDriver commands** +(known upstream regression). When you launch Firefox via this plugin +you'll get: + +> `Marionette: WebDriver:NewSession timed out after 8000ms (Firefox +> 155+ in headless mode is known to not respond to legacy Marionette +> TCP commands; use BiDi via geckodriver instead)` + +Workarounds: +- Install **geckodriver** separately (native binary, outside this + plugin's "no native binaries" rule — install it yourself). +- Use Chrome instead — works without any extras. +- Use Firefox ESR 115 or older (Marionette still works there). + +The Firefox launcher (`lib/cdp-firefox.mjs`) is in place and works +correctly on Firefox versions where Marionette is responsive. + +## Required external tools (user-installed, NOT shipped) + +| Mode | Linux | macOS | Windows | +| --- | --- | --- | --- | +| OS desktop | `grim` / `scrot` + `xdotool` | `screencapture` (built-in) + `cliclick` | PowerShell (built-in) | +| Private Chrome | `google-chrome` / `chromium` | `google-chrome` | `chrome.exe` | +| Inherited Chrome | same + your profile | same + your profile | same + your profile | +| Private Firefox | `firefox` (and optionally `geckodriver`) | same | same | + +The plugin refuses to install any of these and fails fast with a +helpful error if any are missing. + +## OS permissions (Mode 1 only) + +- **macOS**: System Settings → Privacy & Security → Screen Recording + → add your terminal. Accessibility → add your terminal. +- **Linux**: usually no prompt; some Wayland compositors require the + user to grant screen capture explicitly. +- **Windows**: usually no prompt. + +## MCP tools (Mode 1: OS desktop) + +| Tool | Purpose | +| --- | --- | +| `grid_metadata` | First call of a session. Returns seed, palette, screen size, platform info. | +| `screenshot` | Captures the screen + overlays the noise grid. Returns the composite image + structured `{ seed, gridC, palette, w, h, turn, summary_every }`. | +| `click` | Click at `(color1, color2, pos)`. | +| `scroll` | Scroll at `(color1, color2, pos)` by `(dx, dy)`. | +| `drag` | Drag from one addressed pixel to another. | +| `type_text` | Type a string at the current cursor position. | +| `key_combo` | Synthesize a key combination. | +| `cursor_position` | Return the current cursor pixel. | +| `zoom` | Return a noise-free PNG of a region for reading dense text. | +| `retrieve_frame` | Re-inject a previously archived screenshot. | +| `record_summary` | Store a 1–3 sentence summary for a frame. | + +## The noise-grid protocol + +Two random-noise grids are overlaid on the screen: + +- **Layer A**: square cells of size `C = 32 px`, aligned to (0, 0). +- **Layer B**: same cell size, offset by `(-C/2, -C/2)` so its cell + (cx, cy) covers `[cx*C - C/2, (cx+1)*C - C/2)`. Every screen pixel + falls in some Layer B cell. +- Each cell carries one color from a fixed 256-color palette. +- The two layers divide every macro cell of A into **4 sub-cells** + (each 25% of the macro cell). Each sub-cell carries a unique + `(color1, color2)` pair. + +The model addresses any sub-cell on screen by: + +``` +{ + "color1": "#RRGGBB", // Layer A color of the macro cell + "color2": "#RRGGBB", // Layer B color of the overlapping cell + "pos": [dx, dy], // pixel offset from sub-cell center, range [-8, 7] + "button": "left" | "right" | "middle" +} +``` + +The MCP server resolves the tuple to an exact pixel and dispatches the +input event. + +## Trajectory summarization + +Every `screenshot` call increments `state.screenshotCount`. On every +turn that's a multiple of `summary_every` (default 10; override via +`MCODE_COMPUTER_USE_SUMMARY_EVERY`), the next `screenshot` response +carries `structuredContent.summary_request`. The model should: + +1. For each archived turn, call `record_summary({ turn, summary })` + with a 1–3 sentence description. +2. Continue the task. The next `screenshot` returns a **noise-only** + image (no full screen composite). +3. If a later turn requires looking back, call + `retrieve_frame({ turn })` to re-inject the original archived image. + +All archived frames live under `${PLUGIN_DATA}/trajectory/`. The plugin +enforces a 200 MB soft cap (oldest-first eviction). + +## Agent Activity Dashboard + +The plugin serves a small web dashboard on a random localhost port +via `lib/dashboard.mjs`. The URL is printed when the plugin starts +and pushed to the mcode notification channel. Open it in any browser +to see: + +- **Live screen panel**: the model's current view (auto-refresh). +- **Current task**: what the model says it's doing. +- **Action log**: scrolling list of recent tool calls. +- **Trajectory indicator**: countdown to next summary. +- **Permissions panel**: inline "Allow / Decline" buttons for any + consent requests the model raises. + +Plain HTML + JS + SSE — no native window, no install. + +## Private headless Chrome (Mode 2) + +`lib/cdp.mjs` launches a private Chrome under the agent's permissions +and exposes a small JS API: `navigate`, `screenshot`, `click`, +`clickSelector`, `focusSelector`, `typeText`, `evaluate`, `close`. +The agent never needs OS-level screen capture. + +```js +import { launchChrome } from './lib/cdp.mjs'; +const ch = await launchChrome({ width: 1280, height: 720 }); +await ch.navigate('https://example.com'); +const png = await ch.screenshot(); +await ch.focusSelector('input[name="q"]'); +await ch.typeText('hello world'); +await ch.evaluate('document.querySelector("input[name=q]").form.submit()'); +await ch.close(); +``` + +This is the path that works **without** any OS permission, on a +headless server, or in any environment where `screencapture` / +`grim` / xdotool are unavailable. + +## Chromium profile inheritance (Mode 3) + +`lib/profile.mjs` discovers installed Chromium-family profiles +(Chrome, Chromium, Brave, Edge) and copies them to a temp dir. The +copy is then used as the `--user-data-dir` of a private Chrome +instance, so the agent inherits cookies, cache, login state. + +```js +import { listAllProfiles, copyChromiumProfile } from './lib/profile.mjs'; +const all = await listAllProfiles(); +// all.chromium[0].profiles → [ { name: 'Default', hasCookies: true, sizeMB: 432 } ] +const copied = await copyChromiumProfile({ + userDataDir: all.chromium[0].userDataDir, + profileName: 'Default', +}); +// copied.userDataDir is a temp dir owned by the agent +``` + +For Firefox the plugin does NOT use the user's profile. The Firefox +backend always runs a private, agent-owned profile (`mkdtempSync` + +delete on close). This is the explicit design choice — the user keeps +their Firefox private; the agent has its own. + +## Tests + +```bash +cd mcode-computer-use +node --test lib/*.test.mjs +``` + +Covers: +- PRNG determinism. +- Coordinate encoding (encodePixel). +- Round-trip decode within sub-cell tolerance. +- PNG codec encode/decode. +- Overlay / noise-only render produces valid PNG. + +## Disclosure + +- Captures the screen at every Mode 1 `screenshot` call. +- Writes archived PNGs to `${PLUGIN_DATA}/trajectory/` (200 MB cap). +- Writes dashboard events to memory; no remote network calls. +- Profile inheritance: source profile is read-only; copy lives in + temp dir and is removed on close. +- Network: zero from this plugin. +- Native binaries: none shipped. Required CLI tools are user-installed. diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/hooks.json b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/hooks.json new file mode 100644 index 00000000..d45b3499 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/hooks.json @@ -0,0 +1,29 @@ +{ + "$schema": "https://minimax.io/schemas/mcode-hooks/0.1.0/hooks.schema.json", + "hooks": { + "SessionStart": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/session-start.mjs", + "--state", + "${PLUGIN_DATA}/state.json" + ], + "timeout": 5000, + "once": false + } + ], + "PreCompact": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/pre-compact.mjs", + "--state", + "${PLUGIN_DATA}/state.json" + ], + "timeout": 5000, + "once": false + } + ] + } +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/pre-compact.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/pre-compact.mjs new file mode 100644 index 00000000..31e4d1a8 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/pre-compact.mjs @@ -0,0 +1,36 @@ +#!/usr/bin/env node +// PreCompact hook for mcode-computer-use. +// Emits a decision-bearing response so the runtime surfaces a confirmation +// pill before context compaction fires (per proposals/hooks-detailed-spec.md +// § Decision semantics, "ask" value). + +import { argv } from 'node:process'; +import { loadState } from '../../../lib/state.mjs'; + +const args = parseArgs(argv.slice(2)); +const statePath = args.state; + +let count = 0; +if (statePath) { + const cur = await loadState(statePath); + count = cur.screenshotCount || 0; +} + +const body = { + hookSpecificOutput: { + hookEventName: 'PreCompact', + decision: 'ask', + reason: `computer-use session has ${count} screenshot turns; ` + + 'consider summarization via record_summary before compacting.', + }, +}; +process.stdout.write(JSON.stringify(body)); +process.exit(0); + +function parseArgs(args) { + const out = { state: null }; + for (let i = 0; i < args.length; i += 1) { + if (args[i] === '--state') { out.state = args[i + 1]; i += 1; } + } + return out; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/session-start.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/session-start.mjs new file mode 100644 index 00000000..125e7049 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/io.minimax.mcode/hooks/scripts/session-start.mjs @@ -0,0 +1,31 @@ +#!/usr/bin/env node +// SessionStart hook for mcode-computer-use. +// Ensures PLUGIN_DATA exists and resets the screenshot counter. + +import { argv, env } from 'node:process'; +import { mkdir } from 'node:fs/promises'; +import { dirname } from 'node:path'; +import { loadState, saveState } from '../../../lib/state.mjs'; + +const args = parseArgs(argv.slice(2)); +if (!args.state) process.exit(0); +const statePath = args.state; + +await mkdir(dirname(statePath), { recursive: true }); +await mkdir(`${dirname(statePath)}/trajectory`, { recursive: true }); + +const cur = await loadState(statePath); +await saveState(statePath, { + ...cur, + screenshotCount: 0, + sessionStartedAt: new Date().toISOString(), + pid: process.pid, +}); + +function parseArgs(args) { + const out = { state: null }; + for (let i = 0; i < args.length; i += 1) { + if (args[i] === '--state') { out.state = args[i + 1]; i += 1; } + } + return out; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp-firefox.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp-firefox.mjs new file mode 100644 index 00000000..d29111b0 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp-firefox.mjs @@ -0,0 +1,280 @@ +// cdp-firefox.mjs +// +// Firefox backend for the computer-use plugin. +// +// Design: runs PRIVATELY with the agent's own fresh profile, under the +// agent's own process and permissions. The user's Firefox profile is +// never read, copied, or touched in any way. This is the explicit +// Firefox choice for this plugin: "user Firefox, but with the agent's +// permissions" — i.e. the agent has its own Firefox that nobody else +// sees. +// +// Firefox 155 uses the Marionette protocol (port 2828), not the Chrome +// DevTools Protocol. Marionette is a length-prefixed JSON-over-TCP +// protocol — the same protocol Selenium/geckodriver uses. + +import { spawn } from 'node:child_process'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { createConnection } from 'node:net'; + +let nextConn = 0; + +// Find a Firefox binary on PATH. +function findFirefox() { + for (const b of ['firefox', 'firefox-bin', 'firefox-esr']) { + try { + const r = spawn('which', [b], { stdio: ['ignore', 'pipe', 'ignore'] }); + let out = ''; + r.stdout.on('data', (d) => { out += d.toString(); }); + return new Promise((resolve) => { + r.on('close', () => resolve(out.trim() ? { bin: b, path: out.trim() } : null)); + }); + } catch {} + } + return Promise.resolve(null); +} + +// Marionette client: length-prefixed JSON frames over TCP. +// Format: ":\n" (length is ASCII decimal byte count of json) +class MarionetteClient { + constructor() { + this.socket = null; + this.buffer = Buffer.alloc(0); + this.pending = new Map(); // id -> {resolve, reject} + this.events = []; + this.id = 1; + this.sessionId = null; + this._welcomeResolve = null; + this._welcome = new Promise((r) => { this._welcomeResolve = r; }); + } + async connect(host = '127.0.0.1', port = 2828) { + await new Promise((resolve, reject) => { + this.socket = createConnection({ host, port }, resolve); + this.socket.once('error', reject); + }); + this.socket.on('data', (chunk) => this._onData(chunk)); + this.socket.on('close', () => { + for (const { reject: rj } of this.pending.values()) { + rj(new Error('Marionette: socket closed')); + } + this.pending.clear(); + }); + await this._welcome; // wait for the welcome frame + } + _onData(chunk) { + this.buffer = Buffer.concat([this.buffer, chunk]); + while (true) { + const colon = this.buffer.indexOf(0x3A); // ':' + if (colon < 0) return; + const lenStr = this.buffer.slice(0, colon).toString('ascii'); + const len = Number(lenStr); + if (!Number.isInteger(len) || len < 0) { + // Probably the welcome frame (no colon for length? actually it does have ':'). + // If parse fails, drop the byte and continue. + this.buffer = this.buffer.slice(1); + continue; + } + if (this.buffer.length < colon + 1 + len) return; // wait for more + const jsonBuf = this.buffer.slice(colon + 1, colon + 1 + len); + this.buffer = this.buffer.slice(colon + 1 + len); + let msg; + try { msg = JSON.parse(jsonBuf.toString('utf8')); } catch { continue; } + this._dispatch(msg); + } + } + _dispatch(msg) { + if (msg.id !== undefined && this.pending.has(msg.id)) { + const { resolve, reject } = this.pending.get(msg.id); + this.pending.delete(msg.id); + if (msg.error) reject(new Error(`Marionette ${msg.error.error || 'error'}: ${msg.error.message}`)); + else resolve(msg.result || msg); + } else if (msg.applicationType) { + // Welcome frame + this._welcomeResolve(msg); + } else { + this.events.push(msg); + } + } + send(name, args = {}, timeoutMs = 8000) { + const id = this.id++; + const body = { id, name, args }; + if (this.sessionId) body.sessionId = this.sessionId; + const json = JSON.stringify(body); + const frame = `${json.length}:${json}\n`; + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + if (this.pending.has(id)) { + this.pending.delete(id); + reject(new Error(`Marionette: ${name} timed out after ${timeoutMs}ms (Firefox 155+ in headless mode is known to not respond to legacy Marionette TCP commands; use BiDi via geckodriver instead)`)); + } + }, timeoutMs); + this.pending.set(id, { + resolve: (v) => { clearTimeout(timer); resolve(v); }, + reject: (e) => { clearTimeout(timer); reject(e); }, + }); + try { + this.socket.write(frame, (err) => { if (err) reject(err); }); + } catch (e) { + clearTimeout(timer); + this.pending.delete(id); + reject(e); + } + }); + } + drainEvents(predicate) { + const out = []; + for (let i = 0; i < this.events.length; i += 1) { + if (!predicate || predicate(this.events[i])) out.push(this.events[i]); + } + this.events.length = 0; + return out; + } + close() { + try { this.socket.end(); } catch {} + } +} + +export async function launchFirefox({ width = 1280, height = 720 } = {}) { + const ff = await findFirefox(); + if (!ff) throw new Error('Firefox not found on PATH'); + + const profileDir = mkdtempSync(join(tmpdir(), 'mcode-cu-ff-')); + const proc = spawn(ff.path, [ + '--headless', + '--marionette', + '-profile', profileDir, + '--width', String(width), + '--height', String(height), + 'about:blank', + ], { stdio: ['ignore', 'ignore', 'pipe'] }); + let stderr = ''; + proc.stderr.on('data', (d) => { stderr += d.toString(); }); + + // Wait for Marionette port 2828. + await waitForPort(2828, 8000); + const client = new MarionetteClient(); + await client.connect('127.0.0.1', 2828); + + // Start a session. + const session = await client.send('WebDriver:NewSession', { + capabilities: { alwaysMatch: {} }, + }); + client.sessionId = session.value || session.sessionId; + + return { + proc, + profileDir, + width, + height, + client, + browser: 'firefox', + privateProfile: true, + async navigate(url) { + await client.send('WebDriver:Navigate', { url }); + // Wait briefly for load (Marionette doesn't emit load events; sleep). + await new Promise((r) => setTimeout(r, 600)); + }, + async screenshot() { + const r = await client.send('WebDriver:TakeScreenshot', {}); + const data = r.value || r; + // Marionette returns base64-encoded PNG already. + const b64 = typeof data === 'string' ? data : data.data; + return Buffer.from(b64, 'base64'); + }, + async viewport() { + try { + const v = await client.send('WebDriver:ExecuteScript', { + script: 'return {w: window.innerWidth, h: window.innerHeight, dpr: window.devicePixelRatio || 1};', + args: [], + }); + return v.value || v; + } catch { + return { w: width, h: height, dpr: 1 }; + } + }, + async click(x, y, { button = 'left' } = {}) { + await client.send('WebDriver:Actions', { + actions: [{ + type: 'pointer', + id: 'mouse', + parameters: { pointerType: 'mouse' }, + actions: [ + { type: 'pointerMove', x, y, duration: 50 }, + { type: 'pointerDown', button: button === 'right' ? 2 : (button === 'middle' ? 1 : 0) }, + { type: 'pointerUp', button: button === 'right' ? 2 : (button === 'middle' ? 1 : 0) }, + ], + }], + }); + }, + async clickSelector(selector, { button = 'left' } = {}) { + const found = await client.send('WebDriver:FindElement', { using: 'css selector', value: selector }); + const elementId = found.value?.['element-6066-6e73-b8b5-a2b5e2b27d70'] || found.value?.ELEMENT || found.value; + if (!elementId) throw new Error(`no element matches "${selector}"`); + await client.send('WebDriver:ElementClick', { id: elementId }); + return { elementId, selector }; + }, + async focusSelector(selector) { + const found = await client.send('WebDriver:FindElement', { using: 'css selector', value: selector }); + const elementId = found.value?.['element-6066-6e73-b8b5-a2b5e2b27d70'] || found.value?.ELEMENT || found.value; + if (!elementId) throw new Error(`focusSelector: no match for "${selector}"`); + // Fire focus via JS; Marionette's SendKeys requires element interaction. + await client.send('WebDriver:ExecuteScript', { + script: `arguments[0].focus(); return true;`, + args: [{ 'element-6066-6e73-b8b5-a2b5e2b27d70': elementId, ELEMENT: elementId }], + }); + return { elementId }; + }, + async typeText(text, { intervalMs = 0 } = {}) { + for (const ch of text) { + await client.send('WebDriver:Input', { + actions: [{ + type: 'key', + id: 'kbd', + actions: [ + { type: 'keyDown', value: ch }, + { type: 'keyUp', value: ch }, + ], + }], + }); + if (intervalMs > 0) await new Promise((r) => setTimeout(r, intervalMs)); + } + }, + async evaluate(expression) { + const r = await client.send('WebDriver:ExecuteScript', { + script: expression, + args: [], + }); + return r.value ?? r; + }, + async close() { + try { await client.send('WebDriver:Quit', {}); } catch {} + try { client.close(); } catch {} + try { proc.kill('SIGTERM'); } catch {} + await new Promise((r) => setTimeout(r, 250)); + try { rmSync(profileDir, { recursive: true, force: true }); } catch {} + }, + }; +} + +function waitForPort(port, timeoutMs) { + return new Promise((resolve, reject) => { + const start = Date.now(); + const tryOnce = () => { + const sock = createConnection({ host: '127.0.0.1', port }, () => { + sock.end(); + resolve(); + }); + sock.once('error', () => { + sock.destroy(); + if (Date.now() - start > timeoutMs) { + reject(new Error(`Marionette: port ${port} did not open within ${timeoutMs}ms`)); + } else { + setTimeout(tryOnce, 100); + } + }); + }; + tryOnce(); + }); +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp.mjs new file mode 100644 index 00000000..fed33ca2 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/cdp.mjs @@ -0,0 +1,349 @@ +// cdp.mjs +// +// Chrome DevTools Protocol (CDP) backend for the computer-use plugin. +// +// This is an alternative to the OS-level screenshot+input pipeline. It +// launches a private headless Chrome (visible to nobody — no screen +// capture permission needed) and drives it via the standard CDP +// WebSocket protocol. The model sees the page just like any other +// screenshot; clicks and keystrokes are dispatched through Chrome +// itself, not through xdotool/cliclick. +// +// Trade-offs vs the OS-level pipeline: +// + No OS permission required (no Screen Recording / Accessibility). +// + No external CLI tools needed (no xdotool, no cliclick, no grim). +// + Pure-JS implementation; ships inside the plugin. +// + Works on headless servers with no X server. +// - Limited to the Chrome/Firefox page (not the whole desktop). +// - Multiple displays or non-browser apps not visible. +// - Browser launch adds ~1s of overhead per session. +// +// Use the OS-level pipeline when you have permissions and want true +// desktop control; use this CDP backend when you don't. + +import { spawn } from 'node:child_process'; +import { mkdtempSync, mkdirSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +let nextId = 1; + +function findChrome() { + for (const b of ['google-chrome', 'google-chrome-stable', 'chrome', 'chromium', 'chromium-browser']) { + try { + const r = spawn('which', [b], { stdio: ['ignore', 'pipe', 'ignore'] }); + let out = ''; + r.stdout.on('data', (d) => { out += d.toString(); }); + return new Promise((resolve) => { + r.on('close', () => { + const p = out.trim(); + resolve(p ? { bin: b, path: p } : null); + }); + }); + } catch {} + } + return Promise.resolve(null); +} + +async function getWsUrl(port) { + for (let i = 0; i < 50; i += 1) { + try { + const res = await fetch(`http://127.0.0.1:${port}/json/version`); + if (res.ok) { + const j = await res.json(); + return j.webSocketDebuggerUrl; + } + } catch {} + await new Promise((r) => setTimeout(r, 100)); + } + throw new Error(`CDP: Chrome did not open debugging port ${port} within 5s`); +} + +async function getFirstTargetWs(port) { + for (let i = 0; i < 50; i += 1) { + try { + const res = await fetch(`http://127.0.0.1:${port}/json/list`); + if (res.ok) { + const list = await res.json(); + const target = list.find((t) => t.type === 'page') || list[0]; + if (target) return target.webSocketDebuggerUrl; + } + } catch {} + await new Promise((r) => setTimeout(r, 100)); + } + throw new Error(`CDP: no page target found on port ${port} within 5s`); +} + +// Tiny JSON-RPC over WebSocket client. +function rpc(wsUrl) { + return new Promise((resolve, reject) => { + const ws = new WebSocket(wsUrl); + const pending = new Map(); + const events = []; + let readyRes = null; + const ready = new Promise((r) => { readyRes = r; }); + + ws.addEventListener('message', (ev) => { + let msg; + try { msg = JSON.parse(ev.data); } catch { return; } + if (msg.id !== undefined && pending.has(msg.id)) { + const { resolve: r, reject: rj } = pending.get(msg.id); + pending.delete(msg.id); + if (msg.error) rj(new Error(`CDP error ${msg.error.code}: ${msg.error.message}`)); + else r(msg.result); + } else if (msg.method) { + events.push(msg); + } + }); + + ws.addEventListener('open', () => { + readyRes(); + }); + + ws.addEventListener('close', () => { + for (const { reject: rj } of pending.values()) rj(new Error('CDP: socket closed')); + pending.clear(); + }); + + ws.addEventListener('error', (e) => { + reject(new Error(`CDP: socket error: ${e.message || 'unknown'}`)); + }); + + ready.then(() => { + // Hand back the live object once the socket is open. + resolve({ + send(method, params = {}) { + const id = nextId++; + return new Promise((r, rj) => { + pending.set(id, { resolve: r, reject: rj }); + ws.send(JSON.stringify({ id, method, params })); + }); + }, + drainEvents(methodFilter) { + const out = []; + for (let i = 0; i < events.length; i += 1) { + if (!methodFilter || events[i].method === methodFilter) { + out.push(events[i]); + } + } + events.length = 0; + return out; + }, + close() { + try { ws.close(); } catch {} + }, + }); + }).catch(reject); + }); +} + +// Spawn a private headless Chrome and return its CDP control. +export async function launchChrome({ port = 9222, width = 1280, height = 720 } = {}) { + const chrome = await findChrome(); + if (!chrome) { + throw new Error('CDP: no Chrome/Chromium binary found on PATH'); + } + const profileDir = mkdtempSync(join(tmpdir(), 'mcode-cdp-')); + const args = [ + `--remote-debugging-port=${port}`, + `--remote-debugging-address=127.0.0.1`, + '--headless=new', + '--no-sandbox', + '--disable-gpu', + '--disable-dev-shm-usage', + '--no-first-run', + '--no-default-browser-check', + '--disable-extensions', + '--disable-background-networking', + '--disable-sync', + `--user-data-dir=${profileDir}`, + `--window-size=${width},${height}`, + 'about:blank', + ]; + const proc = spawn(chrome.path, args, { stdio: ['ignore', 'ignore', 'pipe'] }); + let stderr = ''; + proc.stderr.on('data', (d) => { stderr += d.toString(); }); + + const wsUrl = await getWsUrl(port); + const targetWsUrl = await getFirstTargetWs(port); + const client = await rpc(targetWsUrl); + + // Enable the domains we use. + await client.send('Page.enable'); + await client.send('Runtime.enable'); + await client.send('DOM.enable'); + await client.send('Input.setIgnoreInputEvents', { ignore: false }); + + return { + proc, + profileDir, + wsUrl, + port, + width, + height, + client, + async navigate(url) { + await client.send('Page.navigate', { url }); + // Wait for loadState 'load' via Page.loadEventFired event. + const ev = await new Promise((resolve) => { + const t = setTimeout(() => resolve(null), 15000); + const interval = setInterval(() => { + const found = client.drainEvents('Page.loadEventFired'); + if (found.length > 0) { + clearTimeout(t); + clearInterval(interval); + resolve(found[0]); + } + }, 50); + }); + // Give the page another moment to settle (image paints etc.). + await new Promise((r) => setTimeout(r, 250)); + return ev; + }, + async screenshot({ format = 'png' } = {}) { + const res = await client.send('Page.captureScreenshot', { format }); + return Buffer.from(res.data, 'base64'); + }, + async viewport() { + const { result } = await client.send('Runtime.evaluate', { + expression: '({w: window.innerWidth, h: window.innerHeight, dpr: window.devicePixelRatio})', + returnByValue: true, + }); + return result.value; + }, + async click(x, y, { button = 'left', clickCount = 1 } = {}) { + const common = { x, y, button, clickCount }; + await client.send('Input.dispatchMouseEvent', { + type: 'mousePressed', ...common, + }); + await client.send('Input.dispatchMouseEvent', { + type: 'mouseReleased', ...common, + }); + }, + // Find an element via a CSS selector and click it at its center. + // More reliable than computing pixel coordinates manually. + async clickSelector(selector, { button = 'left' } = {}) { + const rect = await client.send('Runtime.evaluate', { + expression: ` + (function(){ + var el = document.querySelector(${JSON.stringify(selector)}); + if (!el) return null; + el.scrollIntoView({block: 'center'}); + var r = el.getBoundingClientRect(); + return { x: r.left + r.width/2, y: r.top + r.height/2, + tag: el.tagName, id: el.id, name: el.name }; + })() + `, + returnByValue: true, + }); + if (!rect.result.value) { + throw new Error(`clickSelector: no element matches "${selector}"`); + } + const { x, y } = rect.result.value; + await client.send('Input.dispatchMouseEvent', { + type: 'mousePressed', x, y, button, clickCount: 1, + }); + await client.send('Input.dispatchMouseEvent', { + type: 'mouseReleased', x, y, button, clickCount: 1, + }); + return rect.result.value; + }, + // Find an input/textarea element by selector and focus it via + // element.focus() rather than by clicking. Faster and immune to + // viewport coordinate mismatches. + async focusSelector(selector) { + const r = await client.send('Runtime.evaluate', { + expression: ` + (function(){ + var el = document.querySelector(${JSON.stringify(selector)}); + if (!el) return null; + el.focus(); + return { tag: el.tagName, id: el.id, name: el.name, + active: document.activeElement === el }; + })() + `, + returnByValue: true, + }); + if (!r.result.value) { + throw new Error(`focusSelector: no element matches "${selector}"`); + } + if (!r.result.value.active) { + // Fall back to mouse click at the element center. + const rect = await client.send('Runtime.evaluate', { + expression: ` + (function(){ + var el = document.querySelector(${JSON.stringify(selector)}); + el.scrollIntoView({block: 'center'}); + var r = el.getBoundingClientRect(); + return { x: r.left + r.width/2, y: r.top + r.height/2 }; + })() + `, + returnByValue: true, + }); + const { x, y } = rect.result.value; + await client.send('Input.dispatchMouseEvent', { + type: 'mousePressed', x, y, button: 'left', clickCount: 1, + }); + await client.send('Input.dispatchMouseEvent', { + type: 'mouseReleased', x, y, button: 'left', clickCount: 1, + }); + } + return r.result.value; + }, + async mousemove(x, y) { + await client.send('Input.dispatchMouseEvent', { + type: 'mouseMoved', x, y, button: 'none', + }); + }, + async typeText(text, { intervalMs = 0 } = {}) { + for (const ch of text) { + if (ch === '\n') { + await client.send('Input.dispatchKeyEvent', { + type: 'char', text: '\r', unmodifiedText: '\r', + windowsVirtualKeyCode: 13, + }); + } else { + await client.send('Input.dispatchKeyEvent', { + type: 'char', text: ch, unmodifiedText: ch, + }); + } + if (intervalMs > 0) await new Promise((r) => setTimeout(r, intervalMs)); + } + }, + async keyCombo(keys) { + // Press all modifier keys down, the main key, then release modifiers. + const modifiers = keys.slice(0, -1).map((k) => k.toLowerCase()); + const main = keys[keys.length - 1]; + for (const m of modifiers) { + await client.send('Input.dispatchKeyEvent', { type: 'rawKeyDown', modifiers: modBit(m), key: m }); + } + await client.send('Input.dispatchKeyEvent', { type: 'char', text: main }); + await client.send('Input.dispatchKeyEvent', { type: 'keyUp', modifiers: modBit(main) }); + for (const m of modifiers) { + await client.send('Input.dispatchKeyEvent', { type: 'keyUp', modifiers: modBit(m), key: m }); + } + }, + async evaluate(expression) { + const { result, exceptionDetails } = await client.send('Runtime.evaluate', { + expression, returnByValue: true, awaitPromise: true, + }); + if (exceptionDetails) throw new Error(exceptionDetails.text || 'CDP eval failed'); + return result.value; + }, + async close() { + try { client.close(); } catch {} + try { proc.kill('SIGTERM'); } catch {} + try { rmSync(profileDir, { recursive: true, force: true }); } catch {} + }, + }; +} + +function modBit(name) { + const n = name.toLowerCase(); + let bit = 0; + if (n === 'ctrl' || n === 'control') bit |= 2; + if (n === 'alt' || n === 'option') bit |= 1; + if (n === 'shift') bit |= 8; + if (n === 'meta' || n === 'cmd' || n === 'super') bit |= 4; + return bit; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/dashboard.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/dashboard.mjs new file mode 100644 index 00000000..b2bbe1b7 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/dashboard.mjs @@ -0,0 +1,467 @@ +// dashboard.mjs +// +// Localhost-only HTTP server that serves the Agent Activity Dashboard. +// Uses SSE for live updates and a vanilla HTML/JS frontend — no +// external dependencies, no build step. + +import { createServer } from 'node:http'; +import { hostname } from 'node:os'; + +const DASHBOARD_HTML = ` + + + + + mcode-computer-use — Activity Dashboard + + + +
+

mcode-computer-use Activity

+ +
+
+
+
+

Live screen

+
+
Waiting for first screenshot…
+
+
+
+

Current task

+
+
+ Turn — + Until summary — +
+
+
No task announced yet.
+
+
+
+
+
+

Action log

+
+
    +
    +
    +
    +

    Permissions

    +
    +
    No permission requests yet.
    +
    +
    +
    +
    + + + +`; + +export async function startDashboard({ host = '127.0.0.1', port = 0 } = {}) { + // State and subscribers. + const subs = new Set(); + const pendingPerms = new Map(); // id -> {resolve, reject, timer} + const eventLog = []; // recent events for replay to new clients + const MAX_LOG = 200; + let currentTask = null; + let currentScreen = null; // {data, meta} + let currentTurn = 0; + let currentSummaryEvery = 10; + + function broadcast(ev) { + eventLog.push(ev); + if (eventLog.length > MAX_LOG) eventLog.shift(); + const data = `data: ${JSON.stringify(ev)}\n\n`; + for (const res of subs) { + try { res.write(data); } catch {} + } + } + + function replay(res) { + // Push the latest screen state. + if (currentScreen) { + try { + res.write(`data: ${JSON.stringify({ type: 'screen', data: currentScreen.data, meta: currentScreen.meta })}\n\n`); + } catch {} + } + if (currentTask) { + try { + res.write(`data: ${JSON.stringify({ type: 'task', text: currentTask.text, turn: currentTurn, summaryEvery: currentSummaryEvery })}\n\n`); + } catch {} + } + // Replay recent events. + for (const ev of eventLog) { + try { res.write(`data: ${JSON.stringify(ev)}\n\n`); } catch {} + } + } + + const server = createServer(async (req, res) => { + if (req.method === 'GET' && req.url === '/') { + res.writeHead(200, { 'content-type': 'text/html; charset=utf-8' }); + res.end(DASHBOARD_HTML); + return; + } + if (req.method === 'GET' && req.url === '/events') { + res.writeHead(200, { + 'content-type': 'text/event-stream', + 'cache-control': 'no-cache', + 'connection': 'keep-alive', + }); + subs.add(res); + // Initial state flush + try { + res.write(`: connected\n\n`); + } catch {} + req.on('close', () => subs.delete(res)); + // Replay buffered state to new clients. + replay(res); + return; + } + if (req.method === 'POST' && req.url.startsWith('/permissions/')) { + const id = decodeURIComponent(req.url.slice('/permissions/'.length)); + let body = ''; + req.on('data', (c) => body += c); + req.on('end', () => { + let parsed; + try { parsed = JSON.parse(body); } catch { parsed = {}; } + const decision = !!parsed.decision; + const pending = pendingPerms.get(id); + if (pending) { + pendingPerms.delete(id); + pending.resolve(decision); + } + res.writeHead(200, { 'content-type': 'application/json' }); + res.end(JSON.stringify({ ok: true })); + }); + return; + } + res.writeHead(404).end('not found'); + }); + + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(port, host, () => resolve()); + }); + const addr = server.address(); + const url = `http://${host}:${addr.port}`; + + return { + host, + port: addr.port, + url, + hostname: hostname(), + // Event pushers. + pushScreen(pngBase64, meta = {}) { + currentScreen = { data: pngBase64, meta }; + broadcast({ type: 'screen', data: pngBase64, meta }); + }, + pushTask({ text, turn, summaryEvery }) { + currentTask = { text, turn, summaryEvery }; + if (turn !== undefined) currentTurn = turn; + if (summaryEvery !== undefined) currentSummaryEvery = summaryEvery; + broadcast({ type: 'task', text, turn, summaryEvery }); + }, + pushTool({ tool, summary, kind = 'info', turn, summaryEvery }) { + if (turn !== undefined) currentTurn = turn; + if (summaryEvery !== undefined) currentSummaryEvery = summaryEvery; + broadcast({ type: 'tool', tool, summary, kind, turn, summaryEvery }); + }, + pushSummaryFlash() { + broadcast({ type: 'summary-flash' }); + }, + // Permission request: returns a Promise when user clicks Allow/Decline. + async askPermission({ title, body, timeoutMs = 10 * 60 * 1000 }) { + const id = `p${Date.now()}_${Math.random().toString(36).slice(2, 8)}`; + broadcast({ type: 'perm-request', id, title, body }); + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + pendingPerms.delete(id); + broadcast({ type: 'perm-update', id, status: 'timed_out' }); + resolve(false); + }, timeoutMs); + pendingPerms.set(id, { + resolve: (decision) => { clearTimeout(timer); resolve(decision); }, + reject, + }); + }); + }, + async close() { + for (const res of subs) try { res.end(); } catch {} + subs.clear(); + await new Promise((r) => server.close(r)); + }, + }; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/decode.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/decode.mjs new file mode 100644 index 00000000..ac398248 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/decode.mjs @@ -0,0 +1,77 @@ +// decode.mjs +// +// Canonical direction used by the MCP tools: +// Given a (color1, color2, pos, [button]) tuple, find the absolute pixel +// (x, y) on the screen and dispatch the requested action. + +import { PALETTE } from './noise-grid.mjs'; + +const QUADRANT_OFFSETS = [ + [0, 0], // TL (0) + [0, 1], // BL (1) + [1, 0], // TR (2) + [1, 1], // BR (3) +]; + +// Find every pixel covered by the sub-cell whose (colorA, colorB) matches. +// Returns an array of { x, y } pixel centers — typically four, one per +// macro cell that contains the pair. +function findSubCells(grid, color1, color2) { + const c = grid.gridC; + const half = c / 2; + const matches = []; + const lc1 = color1.toLowerCase(); + const lc2 = color2.toLowerCase(); + for (let cy = 0; cy < grid.rowsA; cy += 1) { + for (let cx = 0; cx < grid.colsA; cx += 1) { + const ia = grid.layerA[cy * grid.colsA + cx]; + const c1 = PALETTE[ia]; + if (c1.toLowerCase() !== lc1) continue; + const macroX = cx * c; + const macroY = cy * c; + // The four sub-cells live at each quadrant. The B cell overlapping + // sub-cell (qx, qy) within macro cell (cx, cy) is (cx+qx, cy+qy), + // because B's cell (cx, cy) covers [cx*c - c/2, cx*c + c/2), and + // the TL sub-cell of A's macro cell (cx, cy) sits at [cx*c, cx*c + c/2) + // which falls inside B cell (cx, cy). + for (let q = 0; q < 4; q += 1) { + const [qx, qy] = QUADRANT_OFFSETS[q]; + const bx = cx + qx; + const by = cy + qy; + if (bx < 0 || by < 0 || bx >= grid.colsB || by >= grid.rowsB) continue; + const ib = grid.layerB[by * grid.colsB + bx]; + const c2 = PALETTE[ib]; + if (c2.toLowerCase() === lc2) { + matches.push({ + macroOrigin: [macroX, macroY], + quadrant: q, + subOrigin: [macroX + qx * half, macroY + qy * half], + }); + } + } + } + } + return matches; +} + +// Resolve (color1, color2, pos) to a single pixel. If multiple sub-cells +// match the color pair, we pick the topmost-leftmost (deterministic). +// If no match, returns null and the caller should report an error. +export function decodeAddress(grid, { color1, color2, pos }) { + const matches = findSubCells(grid, color1, color2); + if (matches.length === 0) return null; + // Sort by subOrigin (top-left first) for determinism. + matches.sort((a, b) => { + if (a.subOrigin[1] !== b.subOrigin[1]) return a.subOrigin[1] - b.subOrigin[1]; + return a.subOrigin[0] - b.subOrigin[0]; + }); + const m = matches[0]; + const half = grid.gridC / 2; + const [px, py] = pos; + // Clamp to sub-cell bounds to be safe. + const dx = Math.max(-half / 2, Math.min(half / 2 - 1, px | 0)); + const dy = Math.max(-half / 2, Math.min(half / 2 - 1, py | 0)); + const x = m.subOrigin[0] + half / 2 + dx; + const y = m.subOrigin[1] + half / 2 + dy; + return { x, y, matches: matches.length, subOrigin: m.subOrigin }; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode-decode.test.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode-decode.test.mjs new file mode 100644 index 00000000..d9ede108 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode-decode.test.mjs @@ -0,0 +1,67 @@ +// Round-trip test for encode/decode (pure functions). +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { buildGrid } from './noise-grid.mjs'; +import { encodePixel } from './encode.mjs'; +import { decodeAddress } from './decode.mjs'; + +test('encode → decode lands in the same sub-cell', () => { + // With 256 colors, the (color1, color2) pair has 65,536 bins, so most + // sub-cells on a 320x240 grid are uniquely addressable. The protocol + // contract is: the decoded pixel is in the same sub-cell as the encoded + // one (within ±half sub-cell tolerance). + const g = buildGrid({ w: 320, h: 240, gridC: 32, seed: 42 }); + const half = g.gridC / 2; // 16 + let tested = 0, exact = 0, subcellMatch = 0; + for (let y = 0; y < 240; y += 3) { + for (let x = 0; x < 320; x += 3) { + const enc = encodePixel(g, x, y); + const dec = decodeAddress(g, { color1: enc.color1, color2: enc.color2, pos: enc.pos }); + assert.ok(dec, `decode should succeed for (${x},${y})`); + if (dec.x === x && dec.y === y) exact += 1; + else if (Math.abs(dec.x - x) < half && Math.abs(dec.y - y) < half) subcellMatch += 1; + tested += 1; + } + } + const success = exact + subcellMatch; + assert.ok(success / tested >= 0.95, + `expected >=95% sub-cell accuracy, got ${(success / tested * 100).toFixed(1)}% (exact=${exact}, subcell=${subcellMatch}, total=${tested})`); +}); + +test('decodeAddress returns null or match for impossible pair', () => { + const g = buildGrid({ w: 320, h: 240, gridC: 32, seed: 42 }); + // The pair "#000000"/"#FFFFFF" should not match anything. + const dec = decodeAddress(g, { color1: '#000000', color2: '#FFFFFF', pos: [0, 0] }); + assert.ok(dec === null || typeof dec.x === 'number'); +}); + +test('encode identifies the correct quadrant', () => { + const g = buildGrid({ w: 64, h: 64, gridC: 32, seed: 1 }); + // Top-left corner of macro cell at (0,0): quadrant 0 (TL). + assert.equal(encodePixel(g, 0, 0).quadrant, 0); + // Bottom-left of (0,0) macro cell: quadrant 1 (BL). + assert.equal(encodePixel(g, 0, 31).quadrant, 1); + // Top-right of (0,0) macro cell: quadrant 2 (TR). + assert.equal(encodePixel(g, 31, 0).quadrant, 2); + // Bottom-right of (0,0) macro cell: quadrant 3 (BR). + assert.equal(encodePixel(g, 31, 31).quadrant, 3); + // Now (32,0) macro cell, TL. + assert.equal(encodePixel(g, 32, 0).quadrant, 0); +}); + +test('decodeAddress clamps pos out of range', () => { + const g = buildGrid({ w: 320, h: 240, gridC: 32, seed: 1 }); + // Pick any valid (color1, color2) from the grid. + let dec; + for (let y = 0; y < 240 && !dec; y += 16) { + for (let x = 0; x < 320 && !dec; x += 16) { + const enc = encodePixel(g, x, y); + dec = decodeAddress(g, { color1: enc.color1, color2: enc.color2, pos: [1000, -1000] }); + if (dec) { + assert.ok(dec.x >= 0 && dec.x < 320); + assert.ok(dec.y >= 0 && dec.y < 240); + } + } + } + assert.ok(dec, 'should have decoded at least once'); +}); diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode.mjs new file mode 100644 index 00000000..4e7d6520 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/encode.mjs @@ -0,0 +1,47 @@ +// encode.mjs +// +// Diagnostic / symmetry helpers: given an (x, y) pixel, return the +// (color1, color2, pos) tuple that the model would use to address it. +// Round-tripped against decode.mjs for tests. + +import { PALETTE, colorsAtPixel } from './noise-grid.mjs'; + +// For a pixel (x, y), compute the canonical sub-cell address. +// pos is the offset within the sub-cell, range [-(C/4), +(C/4)) on each axis. +export function encodePixel(grid, x, y) { + const c = grid.gridC; + const half = c / 2; + + // Sub-cell origin: A's macro cell origin, plus the B offset inside it. + const macroX = Math.floor(x / c) * c; + const macroY = Math.floor(y / c) * c; + // Which sub-cell quadrant (TL, TR, BL, BR) of the macro cell? + // Layer B's grid is offset by (c/2, c/2). So: + // localX = x - macroX + // localY = y - macroY + // If localX < c/2 we are in the LEFT half of A's macro cell, otherwise RIGHT. + // But B's offset means B's cell boundary sits at c/2; the sub-cell + // containing (x, y) is determined by which of A's 4 quadrants we are in. + const localX = x - macroX; + const localY = y - macroY; + const quadrant = (localX >= half ? 2 : 0) | (localY >= half ? 1 : 0); + + // Sub-cell origin in pixel coordinates: + const subOriginX = macroX + (quadrant & 2 ? half : 0); + const subOriginY = macroY + (quadrant & 1 ? half : 0); + + const { color1, color2 } = colorsAtPixel(grid, x, y); + + // pos is (x - subOriginX - half/2, y - subOriginY - half/2), + // i.e. offset from sub-cell center. + const pos = [x - subOriginX - half / 2, y - subOriginY - half / 2]; + + return { + color1, + color2, + pos, + quadrant, + subOrigin: [subOriginX, subOriginY], + macroOrigin: [macroX, macroY], + }; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/input.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/input.mjs new file mode 100644 index 00000000..073204fc --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/input.mjs @@ -0,0 +1,277 @@ +// input.mjs +// +// Inject mouse and keyboard events using the platform-specific CLI tool. + +import { spawnSync } from 'node:child_process'; + +export async function click(plan, x, y, { button = 'left', modifiers = [] } = {}) { + if (!plan.ready) { + const err = new Error('computer-use: input tools not ready'); + err.code = 'TOOLS_MISSING'; + err.hint = plan.needs.join('; '); + throw err; + } + const k = plan.input.kind; + if (k === 'cliclick') { + // cliclick p:, ... ; buttons: c: left, t: right, m: middle. + const btn = button === 'left' ? 'c' : button === 'right' ? 't' : 'm'; + const r = spawnSync('cliclick', [`p:${x},${y}`, `m:${x},${y}`, `${btn}:${x},${y}`], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`cliclick failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'xdotool') { + const mod = modifiersToXdotool(modifiers); + const btn = button === 'left' ? 1 : button === 'right' ? 3 : 2; + const r = spawnSync('xdotool', [...mod, 'mousemove', '--sync', String(x), String(y), 'click', String(btn)], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`xdotool click failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'ydotool') { + // ydotool uses --button 0x110=left, 0x111=right, 0x112=middle (BTN_MOUSE). + const btn = button === 'left' ? '0x110' : button === 'right' ? '0x111' : '0x112'; + const r = spawnSync('ydotool', ['mousemove', '-a', '-x', String(x), '-y', String(y), 'click', btn], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`ydotool click failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'powershell-sendinput') { + // PowerShell SendInput via .NET. We translate a single click. + const btnFlag = button === 'left' ? 0x0002 : button === 'right' ? 0x0008 : 0x0020; + const downFlag = btnFlag; + const upFlag = btnFlag * 2; // MOUSEEVENTF_LEFTUP = 0x0004 for left, etc. + const ps = ` +Add-Type @" +using System; +using System.Runtime.InteropServices; +public class W { + [DllImport("user32.dll")] public static extern bool SetCursorPos(int X, int Y); + [DllImport("user32.dll")] public static extern void mouse_event(int flags, int dx, int dy, int dw, IntPtr ext); +} +"@ +[W]::SetCursorPos(${x}, ${y}) +[W]::mouse_event(${downFlag}, 0, 0, 0, [IntPtr]::Zero) +[W]::mouse_event(${upFlag}, 0, 0, 0, [IntPtr]::Zero) +`; + const r = spawnSync('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', ps], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`PowerShell click failed: ${r.stderr || r.stdout}`); + return; + } + throw new Error(`unsupported input kind ${k}`); +} + +export async function scroll(plan, x, y, dx, dy) { + if (!plan.ready) { + const err = new Error('computer-use: input tools not ready'); + err.code = 'TOOLS_MISSING'; + throw err; + } + const k = plan.input.kind; + if (k === 'cliclick') { + // cliclick wheel events: there's no direct wheel; simulate via arrow keys + // or skip on platforms without it. For minimal support we run an AppleScript. + // However, cliclick does NOT support wheel; we use osascript fallback. + const r = spawnSync('osascript', ['-e', + `tell application "System Events" to scroll {x:${x}, y:${y}} by ${dy}`], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`osascript scroll failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'xdotool') { + const button = dy > 0 ? 5 : (dy < 0 ? 4 : (dx > 0 ? 7 : 6)); + const r = spawnSync('xdotool', ['mousemove', '--sync', String(x), String(y), + 'click', String(button)], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`xdotool scroll failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'ydotool') { + // ydotool doesn't model scroll directly; emulate with keyboard arrows for vertical. + // For simplicity we run a sequence of PageUp/PageDown if dy != 0. + if (dy !== 0) { + const key = dy > 0 ? 'PageDown' : 'PageUp'; + const steps = Math.min(Math.abs(dy), 20); + for (let i = 0; i < steps; i += 1) { + spawnSync('ydotool', ['key', key], { encoding: 'utf8' }); + } + } + return; + } + if (k === 'powershell-sendinput') { + // WHEEL_DELTA = 120, positive = up + const delta = -dy * 120; + const ps = ` +Add-Type @" +using System; +using System.Runtime.InteropServices; +public class W2 { + [DllImport("user32.dll")] public static extern bool SetCursorPos(int X, int Y); + [DllImport("user32.dll")] public static extern void mouse_event(int flags, int dx, int dy, int dw, IntPtr ext); +} +"@ +[W2]::SetCursorPos(${x}, ${y}) +[W2]::mouse_event(0x0800, 0, 0, ${delta}, [IntPtr]::Zero) +`; + const r = spawnSync('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', ps], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`PowerShell scroll failed: ${r.stderr || r.stdout}`); + return; + } + throw new Error(`unsupported input kind ${k}`); +} + +export async function drag(plan, from, to, { button = 'left' } = {}) { + const k = plan.input.kind; + if (k === 'xdotool') { + const btn = button === 'left' ? 1 : button === 'right' ? 3 : 2; + const r = spawnSync('xdotool', ['mousemove', '--sync', String(from[0]), String(from[1]), + 'mousedown', String(btn), + 'mousemove', '--sync', String(to[0]), String(to[1]), + 'mouseup', String(btn)], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`xdotool drag failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'cliclick') { + const btn = button === 'left' ? 'c' : button === 'right' ? 't' : 'm'; + const r = spawnSync('cliclick', + [`dd:${btn}`, `m:${from[0]},${from[1]}`, `du:${btn}`, `m:${to[0]},${to[1]}`], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`cliclick drag failed: ${r.stderr || r.stdout}`); + return; + } + // Fallback: sequence of move + click. + await click(plan, from[0], from[1], { button }); + await new Promise((r) => setTimeout(r, 50)); + await click(plan, to[0], to[1], { button }); +} + +export async function typeText(plan, text, { intervalMs = 0 } = {}) { + const k = plan.input.kind; + if (k === 'xdotool') { + // xdotool type handles unicode but special chars need a window id; default is fine. + const r = spawnSync('xdotool', ['type', '--delay', String(intervalMs), '--clearmodifiers', text], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`xdotool type failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'cliclick') { + // cliclick t:"text" types a string. Newlines and tabs need escape. + const safe = text.replace(/\\/g, '\\\\').replace(/"/g, '\\"'); + const r = spawnSync('cliclick', [`t:"${safe}"`], { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`cliclick type failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'powershell-sendinput') { + // Use SendInput with virtual keys; basic printable chars only. + const ps = ` +Add-Type -AssemblyName System.Windows.Forms +[System.Windows.Forms.SendKeys]::SendWait($([char]34) + ('${text.replace(/'/g, "''")}' ) + $([char]34)) +`; + // Actually simpler: pipe the text via clipboard + Ctrl+V + const clip = ` +Add-Type -AssemblyName System.Windows.Forms +[System.Windows.Forms.Clipboard]::SetText('${text.replace(/'/g, "''")}') +`; + spawnSync('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', clip], + { encoding: 'utf8' }); + await keyCombo(plan, ['ctrl', 'v']); + return; + } + throw new Error(`unsupported input kind ${k}`); +} + +export async function keyCombo(plan, keys) { + const k = plan.input.kind; + if (k === 'xdotool') { + const r = spawnSync('xdotool', ['key', keys.join('+')], { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`xdotool key failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'cliclick') { + // cliclick uses key names like "tab", "enter", "f1"... limited chord support. + const r = spawnSync('cliclick', ['kp:' + keys.join('-')], { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`cliclick key failed: ${r.stderr || r.stdout}`); + return; + } + if (k === 'powershell-sendinput') { + const map = { + ctrl: '^', alt: '%', shift: '+', cmd: '^', + enter: '{ENTER}', tab: '{TAB}', esc: '{ESC}', escape: '{ESC}', + backspace: '{BS}', delete: '{DEL}', space: ' ', + up: '{UP}', down: '{DOWN}', left: '{LEFT}', right: '{RIGHT}', + home: '{HOME}', end: '{END}', pageup: '{PGUP}', pagedown: '{PGDN}', + }; + const seq = keys.map((k2) => map[k2.toLowerCase()] || k2).join(''); + const ps = ` +Add-Type -AssemblyName System.Windows.Forms +[System.Windows.Forms.SendKeys]::SendWait('${seq.replace(/'/g, "''")}') +`; + const r = spawnSync('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', ps], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`PowerShell key failed: ${r.stderr || r.stdout}`); + return; + } + throw new Error(`unsupported input kind ${k}`); +} + +export async function cursorPosition(plan) { + const k = plan.input.kind; + if (k === 'xdotool') { + const r = spawnSync('xdotool', ['getmouselocation', '--shell'], { encoding: 'utf8' }); + if (r.status !== 0) return null; + const out = {}; + for (const line of r.stdout.split('\n')) { + const [k2, v] = line.split('='); + if (k2 && v) out[k2.trim()] = Number(v); + } + return out; + } + if (k === 'cliclick') { + const r = spawnSync('cliclick', ['p'], { encoding: 'utf8' }); + // Output looks like "123,456" or "error: ..." + if (r.status !== 0) return null; + const m = r.stdout.trim().match(/(-?\d+),(-?\d+)/); + if (m) return { X: Number(m[1]), Y: Number(m[2]) }; + return null; + } + if (k === 'powershell-sendinput') { + const ps = ` +Add-Type @" +using System; +using System.Runtime.InteropServices; +public class W3 { + [StructLayout(LayoutKind.Sequential)] public struct P { public int X, Y; } + [DllImport("user32.dll")] public static extern bool GetCursorPos(out P p); +} +"@ +$p = New-Object W3+P +[W3]::GetCursorPos([ref]$p) | Out-Null +"X=$($p.X) Y=$($p.Y)" +`; + const r = spawnSync('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', ps], + { encoding: 'utf8' }); + if (r.status !== 0) return null; + const out = {}; + for (const part of r.stdout.trim().split(/\s+/)) { + const [k2, v] = part.split('='); + if (k2 && v) out[k2.trim()] = Number(v); + } + return out; + } + return null; +} + +function modifiersToXdotool(modifiers) { + const out = []; + for (const m of modifiers) { + const l = m.toLowerCase(); + if (l === 'ctrl' || l === 'control') out.push('ctrl'); + else if (l === 'alt' || l === 'option') out.push('alt'); + else if (l === 'shift') out.push('shift'); + else if (l === 'super' || l === 'cmd' || l === 'meta' || l === 'win') out.push('super'); + } + return out; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.mjs new file mode 100644 index 00000000..ddc4de4f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.mjs @@ -0,0 +1,166 @@ +// noise-grid.mjs +// +// Deterministic two-layer random-noise grid for the mcode-computer-use +// addressing protocol. +// +// Layer A: square cells of size C aligned to (0, 0). +// Layer B: same cell size, offset by (C/2, C/2). +// +// Each cell carries one color drawn from a fixed 16-color palette. The +// choice is seeded from a per-session PRNG so a session sees the same +// grid across turns but a new session sees a fresh grid. +// +// Pure functions; no I/O; fully unit-testable. + +// 256-color palette derived from HSV space. With 256 colors, the +// (colorA, colorB) pair space has 65,536 bins, which is far more than +// the sub-cell count on any realistic screen (1500–4000 sub-cells at +// 1280x720). Expected number of pair collisions per pair by birthday +// paradox is roughly (n^2) / (2 * 65536), which for n=3600 is ~99 +// collisions across the whole screen — most pairs are unique. +// +// The plan originally targeted 16 colors. That was a deliberate +// trade-off in the visual layer (humans can distinguish 16 hues +// reliably), but for the model addressing protocol it caused too many +// sub-cell collisions. We split the difference: 256 hues is still +// distinguishable to a vision model that has been pre-trained on +// diverse imagery, and the cell sizes (32 px) make the color cells +// large enough that JPEG/PNG compression does not collapse them. +export const PALETTE = (() => { + const out = []; + for (let i = 0; i < 256; i += 1) { + const h = (i * 360 / 256) | 0; + // Vary S/V a bit so adjacent hues stay distinct under compression. + const s = (i & 1) ? 88 : 75; + const v = (i & 2) ? 92 : 80; + out.push(hsvToHex(h, s, v)); + } + return out; +})(); + +function hsvToHex(h, s, v) { + // h in [0,360), s and v in [0,100]. + const c = (v / 100) * (s / 100); + const hh = h / 60; + const x = c * (1 - Math.abs((hh % 2) - 1)); + let r = 0, g = 0, b = 0; + if (0 <= hh && hh < 1) { r = c; g = x; b = 0; } + else if (1 <= hh && hh < 2) { r = x; g = c; b = 0; } + else if (2 <= hh && hh < 3) { r = 0; g = c; b = x; } + else if (3 <= hh && hh < 4) { r = 0; g = x; b = c; } + else if (4 <= hh && hh < 5) { r = x; g = 0; b = c; } + else { r = c; g = 0; b = x; } + const m = (v / 100) - c; + const to255 = (v2) => Math.max(0, Math.min(255, Math.round((v2 + m) * 255))); + return '#' + + to255(r).toString(16).padStart(2, '0').toUpperCase() + + to255(g).toString(16).padStart(2, '0').toUpperCase() + + to255(b).toString(16).padStart(2, '0').toUpperCase(); +} + +// 32-bit xorshift PRNG. Tiny, deterministic, good-enough for grid layout. +function makePrng(seed) { + let s = (seed >>> 0) || 0x9E3779B9; + return function next() { + s ^= s << 13; s >>>= 0; + s ^= s >>> 17; + s ^= s << 5; s >>>= 0; + return s >>> 0; + }; +} + +// Hash a string seed into a 32-bit int. +export function hashSeed(seedStr) { + let h = 2166136261; + for (let i = 0; i < seedStr.length; i += 1) { + h ^= seedStr.charCodeAt(i); + h = Math.imul(h, 16777619); + } + return h >>> 0; +} + +// Build the two layers. Returns: +// { +// seed, gridC, +// layerA: Int32Array of palette indices, length colsA*rowsA, +// layerB: Int32Array of palette indices, length colsB*rowsB, +// colsA, rowsA, colsB, rowsB, w, h, +// } +// +// Layer A is aligned to (0, 0). Layer B is conceptually offset by +// (-c/2, -c/2), so its cell (cx, cy) covers +// [cx*c - c/2, (cx+1)*c - c/2) x [cy*c - c/2, (cy+1)*c - c/2). +// This guarantees every screen pixel in [0, w) x [0, h) has a Layer-B +// color. Layer B therefore needs colsB = colsA + 1 / rowsB = rowsA + 1 +// cells in the worst case so the right/bottom edge is fully covered. +export function buildGrid({ w, h, gridC = 32, seed }) { + if (!Number.isFinite(w) || !Number.isFinite(h) || w <= 0 || h <= 0) { + throw new Error(`buildGrid: invalid dimensions ${w}x${h}`); + } + if (!Number.isInteger(gridC) || gridC < 4) { + throw new Error(`buildGrid: gridC must be an integer >= 4, got ${gridC}`); + } + const seedNum = (typeof seed === 'number') + ? (seed >>> 0) + : hashSeed(String(seed)); + const prng = makePrng(seedNum); + + const colsA = Math.ceil(w / gridC); + const rowsA = Math.ceil(h / gridC); + // For B cell at (cx, cy), its right edge is (cx+1)*c - c/2. + // To cover pixel x = w-1 we need (cx+1)*c - c/2 > w-1, i.e. cx >= (w-1 + c/2)/c. + // The smallest such cx is ceil((w-1 + c/2)/c). Add a small safety margin. + const colsB = Math.ceil((w - 1 + gridC / 2) / gridC) + 1; + const rowsB = Math.ceil((h - 1 + gridC / 2) / gridC) + 1; + + const layerA = new Int32Array(colsA * rowsA); + const layerB = new Int32Array(colsB * rowsB); + + for (let i = 0; i < layerA.length; i += 1) { + layerA[i] = prng() % PALETTE.length; + } + for (let i = 0; i < layerB.length; i += 1) { + layerB[i] = prng() % PALETTE.length; + } + + return { + seed: seedNum, + gridC, + layerA, + layerB, + colsA, rowsA, colsB, rowsB, + w, h, + }; +} + +// Look up the palette index at a screen pixel for a given layer. +// Returns -1 if the pixel is outside the layer's coverage. +export function layerAtPixel(grid, layer, x, y) { + const c = grid.gridC; + if (layer === 'A') { + if (x < 0 || y < 0 || x >= grid.w || y >= grid.h) return -1; + const cx = Math.floor(x / c); + const cy = Math.floor(y / c); + if (cx >= grid.colsA || cy >= grid.rowsA) return -1; + return grid.layerA[cy * grid.colsA + cx]; + } + // B cell (cx, cy) covers [cx*c - c/2, (cx+1)*c - c/2). + // Inverse: cx = floor((x + c/2) / c). + if (x < 0 || y < 0 || x >= grid.w || y >= grid.h) return -1; + const cx = Math.floor((x + c / 2) / c); + const cy = Math.floor((y + c / 2) / c); + if (cx < 0 || cy < 0 || cx >= grid.colsB || cy >= grid.rowsB) return -1; + return grid.layerB[cy * grid.colsB + cx]; +} + +// Return the (colorA, colorB) pair for a screen pixel as hex strings. +export function colorsAtPixel(grid, x, y) { + const ia = layerAtPixel(grid, 'A', x, y); + const ib = layerAtPixel(grid, 'B', x, y); + return { + color1: ia >= 0 ? PALETTE[ia] : null, + color2: ib >= 0 ? PALETTE[ib] : null, + indexA: ia, + indexB: ib, + }; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.test.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.test.mjs new file mode 100644 index 00000000..875156af --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/noise-grid.test.mjs @@ -0,0 +1,62 @@ +// Unit tests for noise-grid.mjs (pure functions). +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { buildGrid, hashSeed, PALETTE, layerAtPixel, colorsAtPixel } from './noise-grid.mjs'; + +test('hashSeed is deterministic', () => { + assert.equal(hashSeed('hello'), hashSeed('hello')); + assert.notEqual(hashSeed('hello'), hashSeed('hello!')); +}); + +test('buildGrid throws on bad dimensions', () => { + assert.throws(() => buildGrid({ w: 0, h: 100 }), /invalid dimensions/); + assert.throws(() => buildGrid({ w: -1, h: 100 }), /invalid dimensions/); + assert.throws(() => buildGrid({ w: 100 }), /invalid dimensions/); +}); + +test('buildGrid throws on bad gridC', () => { + assert.throws(() => buildGrid({ w: 100, h: 100, gridC: 0 }), /gridC/); + assert.throws(() => buildGrid({ w: 100, h: 100, gridC: 3.5 }), /gridC/); +}); + +test('buildGrid is deterministic for a numeric seed', () => { + const a = buildGrid({ w: 320, h: 240, gridC: 32, seed: 42 }); + const b = buildGrid({ w: 320, h: 240, gridC: 32, seed: 42 }); + assert.deepEqual(Array.from(a.layerA), Array.from(b.layerA)); + assert.deepEqual(Array.from(a.layerB), Array.from(b.layerB)); +}); + +test('buildGrid differs across seeds', () => { + const a = buildGrid({ w: 320, h: 240, gridC: 32, seed: 1 }); + const b = buildGrid({ w: 320, h: 240, gridC: 32, seed: 2 }); + assert.notDeepEqual(Array.from(a.layerA), Array.from(b.layerA)); +}); + +test('layerAtPixel returns -1 outside coverage', () => { + const g = buildGrid({ w: 100, h: 100, gridC: 32, seed: 7 }); + assert.equal(layerAtPixel(g, 'A', -1, 0), -1); + assert.equal(layerAtPixel(g, 'B', -1, 0), -1); +}); + +test('layerAtPixel covers both layers', () => { + const g = buildGrid({ w: 320, h: 240, gridC: 32, seed: 7 }); + // Take a few sample points; at least one in each macro cell should resolve. + let sawA = 0, sawB = 0; + for (let y = 0; y < 320; y += 8) { + for (let x = 0; x < 640; x += 8) { + if (layerAtPixel(g, 'A', x, y) >= 0) sawA += 1; + if (layerAtPixel(g, 'B', x, y) >= 0) sawB += 1; + } + } + assert.ok(sawA > 100, 'expected many A hits'); + assert.ok(sawB > 100, 'expected many B hits'); +}); + +test('colorsAtPixel returns palette strings', () => { + const g = buildGrid({ w: 320, h: 240, gridC: 32, seed: 9 }); + const c = colorsAtPixel(g, 100, 100); + assert.ok(PALETTE.includes(c.color1)); + assert.ok(PALETTE.includes(c.color2)); + assert.equal(c.indexA, PALETTE.indexOf(c.color1)); + assert.equal(c.indexB, PALETTE.indexOf(c.color2)); +}); diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.mjs new file mode 100644 index 00000000..ef46b537 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.mjs @@ -0,0 +1,128 @@ +// overlay.mjs +// +// Compose a screenshot (PNG byte buffer) with the two noise layers at low +// alpha, returning a PNG byte buffer. Pure given inputs. +// +// We do pure-JS PNG decoding and encoding via pngjs (no native binaries). +// The bundled encoder/decoder is implemented in lib/png-pure.mjs. + +import { PALETTE } from './noise-grid.mjs'; +import { Png } from './png-pure.mjs'; + +function hexToRgb(hex) { + const h = hex.replace('#', ''); + return [ + parseInt(h.slice(0, 2), 16), + parseInt(h.slice(2, 4), 16), + parseInt(h.slice(4, 6), 16), + ]; +} + +// Compose an overlay. `pngBytes` is a Buffer with the screenshot PNG. +// `alpha` is 0..1; default 0.22 (the plan calls for low-alpha overlay). +export async function composeOverlay(pngBytes, grid, { alpha = 0.22 } = {}) { + const src = await Png.decode(pngBytes); + const { width: w, height: h, data: srcRgba } = src; + + // Use a fresh copy so we don't mutate the caller's buffer. + const out = Buffer.from(srcRgba); + + const c = grid.gridC; + const half = c / 2; + + // Layer A: aligned to (0, 0). For each cell, blend its color. + for (let cy = 0; cy < grid.rowsA; cy += 1) { + for (let cx = 0; cx < grid.colsA; cx += 1) { + const ia = grid.layerA[cy * grid.colsA + cx]; + const [r, g, b] = hexToRgb(PALETTE[ia]); + const x0 = cx * c; + const y0 = cy * c; + const x1 = Math.min(x0 + c, w); + const y1 = Math.min(y0 + c, h); + blendRect(out, w, x0, y0, x1, y1, r, g, b, alpha); + } + } + + // Layer B: B cell (cx, cy) covers [cx*c - c/2, (cx+1)*c - c/2). + // The first B cell (0,0) starts at (-c/2, -c/2), so half of it is + // off-screen on the top-left. We clip to screen bounds. + for (let cy = 0; cy < grid.rowsB; cy += 1) { + for (let cx = 0; cx < grid.colsB; cx += 1) { + const ib = grid.layerB[cy * grid.colsB + cx]; + const [r, g, b] = hexToRgb(PALETTE[ib]); + const x0 = cx * c - half; + const y0 = cy * c - half; + const x1 = Math.min(x0 + c, w); + const y1 = Math.min(y0 + c, h); + if (x1 <= 0 || y1 <= 0 || x0 >= w || y0 >= h) continue; + const cx0 = Math.max(0, x0); + const cy0 = Math.max(0, y0); + // Use a slightly lower alpha for Layer B so both are visible. + blendRect(out, w, cx0, cy0, x1, y1, r, g, b, alpha * 0.85); + } + } + + return await Png.encode({ width: w, height: h, data: out }); +} + +// Build a noise-only image at the same dimensions as the grid. Used for +// the "compact mode" returned after summarization. +export async function renderNoiseOnly(grid, { alpha = 0.55 } = {}) { + const w = grid.w; + const h = grid.h; + const buf = Buffer.alloc(w * h * 4); + + const c = grid.gridC; + const half = c / 2; + + for (let cy = 0; cy < grid.rowsA; cy += 1) { + for (let cx = 0; cx < grid.colsA; cx += 1) { + const ia = grid.layerA[cy * grid.colsA + cx]; + const [r, g, b] = hexToRgb(PALETTE[ia]); + const x0 = cx * c; + const y0 = cy * c; + const x1 = Math.min(x0 + c, w); + const y1 = Math.min(y0 + c, h); + paintRect(buf, w, x0, y0, x1, y1, r, g, b, alpha); + } + } + for (let cy = 0; cy < grid.rowsB; cy += 1) { + for (let cx = 0; cx < grid.colsB; cx += 1) { + const ib = grid.layerB[cy * grid.colsB + cx]; + const [r, g, b] = hexToRgb(PALETTE[ib]); + const x0 = cx * c - half; + const y0 = cy * c - half; + const x1 = Math.min(x0 + c, w); + const y1 = Math.min(y0 + c, h); + if (x1 <= 0 || y1 <= 0 || x0 >= w || y0 >= h) continue; + paintRect(buf, w, Math.max(0, x0), Math.max(0, y0), x1, y1, r, g, b, alpha * 0.85); + } + } + return await Png.encode({ width: w, height: h, data: buf }); +} + +function blendRect(buf, w, x0, y0, x1, y1, r, g, b, alpha) { + for (let y = y0; y < y1; y += 1) { + const row = y * w * 4; + for (let x = x0; x < x1; x += 1) { + const i = row + x * 4; + buf[i] = (buf[i] * (1 - alpha) + r * alpha) | 0; + buf[i + 1] = (buf[i + 1] * (1 - alpha) + g * alpha) | 0; + buf[i + 2] = (buf[i + 2] * (1 - alpha) + b * alpha) | 0; + // alpha channel unchanged + } + } +} + +function paintRect(buf, w, x0, y0, x1, y1, r, g, b, alpha) { + for (let y = y0; y < y1; y += 1) { + const row = y * w * 4; + for (let x = x0; x < x1; x += 1) { + const i = row + x * 4; + buf[i] = (r * alpha) | 0; + buf[i + 1] = (g * alpha) | 0; + buf[i + 2] = (b * alpha) | 0; + buf[i + 3] = 255; + } + } +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.test.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.test.mjs new file mode 100644 index 00000000..dfd2aba6 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/overlay.test.mjs @@ -0,0 +1,28 @@ +// Smoke test for overlay.mjs: round-trip a 64x64 black PNG through the +// noise overlay and verify it's still a valid PNG of the same size. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { buildGrid } from './noise-grid.mjs'; +import { composeOverlay, renderNoiseOnly } from './overlay.mjs'; +import { Png } from './png-pure.mjs'; + +test('composeOverlay produces same-dimension PNG', async () => { + const w = 64, h = 64; + const blank = await Png.encode({ + width: w, height: h, + data: Buffer.alloc(w * h * 4, 0xFF), // white + }); + const g = buildGrid({ w, h, gridC: 32, seed: 5 }); + const out = await composeOverlay(blank, g, { alpha: 0.2 }); + const dec = await Png.decode(out); + assert.equal(dec.width, w); + assert.equal(dec.height, h); +}); + +test('renderNoiseOnly produces valid PNG', async () => { + const g = buildGrid({ w: 128, h: 96, gridC: 32, seed: 5 }); + const out = await renderNoiseOnly(g, { alpha: 0.5 }); + const dec = await Png.decode(out); + assert.equal(dec.width, 128); + assert.equal(dec.height, 96); +}); diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/platform.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/platform.mjs new file mode 100644 index 00000000..214d7b11 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/platform.mjs @@ -0,0 +1,77 @@ +// platform.mjs +// +// Detects the host OS and reports which external CLI tools the plugin +// needs. Also performs a one-time `which`-style probe at session start so +// we fail fast with a friendly message if anything is missing. + +import { spawnSync } from 'node:child_process'; + +export const HOST_OS = (() => { + switch (process.platform) { + case 'darwin': return 'macos'; + case 'linux': return 'linux'; + case 'win32': return 'windows'; + default: return process.platform; + } +})(); + +function which(bin) { + const cmd = process.platform === 'win32' ? 'where' : 'which'; + const r = spawnSync(cmd, [bin], { encoding: 'utf8' }); + return r.status === 0; +} + +// Pick the best screenshot tool available on Linux. +// Order: grim (Wayland) > scrot > gnome-screenshot > import > spectacle. +function pickLinuxScreenshot() { + if (which('grim')) return 'grim'; + if (which('scrot')) return 'scrot'; + if (which('gnome-screenshot')) return 'gnome-screenshot'; + if (which('import')) return 'import'; + if (which('spectacle')) return 'spectacle'; + return null; +} + +function pickLinuxInput() { + if (which('xdotool')) return 'xdotool'; + if (which('ydotool')) return 'ydotool'; + return null; +} + +export function detect() { + const plan = { + os: HOST_OS, + screenshot: null, + input: null, + needs: [], + }; + if (HOST_OS === 'macos') { + plan.screenshot = { kind: 'builtin', bin: 'screencapture' }; + if (which('cliclick')) { + plan.input = { kind: 'cliclick', bin: 'cliclick' }; + } else { + plan.input = null; + plan.needs.push('cliclick (brew install cliclick)'); + } + } else if (HOST_OS === 'linux') { + const ss = pickLinuxScreenshot(); + if (ss) { + plan.screenshot = { kind: ss, bin: ss }; + } else { + plan.needs.push('one of: grim (Wayland) / scrot / gnome-screenshot / import (ImageMagick) / spectacle'); + } + const ip = pickLinuxInput(); + if (ip) { + plan.input = { kind: ip, bin: ip }; + } else { + plan.needs.push('xdotool (X11) or ydotool (Wayland)'); + } + } else if (HOST_OS === 'windows') { + plan.screenshot = { kind: 'powershell-graphics', bin: 'powershell' }; + plan.input = { kind: 'powershell-sendinput', bin: 'powershell' }; + } else { + plan.needs.push(`unsupported platform: ${HOST_OS}`); + } + plan.ready = plan.needs.length === 0 && plan.screenshot && plan.input; + return plan; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.mjs new file mode 100644 index 00000000..c4c8913c --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.mjs @@ -0,0 +1,186 @@ +// png-pure.mjs +// +// Minimal PNG encoder and decoder for RGBA8 (color type 6, bit depth 8). +// Pure JavaScript; only uses node:zlib for inflate/deflate. No external +// dependencies — this keeps the plugin free of native binaries per the +// security model. + +import { deflateSync, inflateSync, constants } from 'node:zlib'; + +const PNG_SIG = Buffer.from([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]); + +// CRC-32 (PNG variant). Precomputed table. +const CRC_TABLE = (() => { + const t = new Uint32Array(256); + for (let n = 0; n < 256; n += 1) { + let c = n; + for (let k = 0; k < 8; k += 1) { + c = (c & 1) ? (0xEDB88320 ^ (c >>> 1)) : (c >>> 1); + } + t[n] = c >>> 0; + } + return t; +})(); + +function crc32(buf) { + let c = 0xFFFFFFFF; + for (let i = 0; i < buf.length; i += 1) { + c = CRC_TABLE[(c ^ buf[i]) & 0xFF] ^ (c >>> 8); + } + return (c ^ 0xFFFFFFFF) >>> 0; +} + +function chunk(type, data) { + const len = Buffer.alloc(4); + len.writeUInt32BE(data.length, 0); + const typeBuf = Buffer.from(type, 'ascii'); + const crcBuf = Buffer.alloc(4); + crcBuf.writeUInt32BE(crc32(Buffer.concat([typeBuf, data])), 0); + return Buffer.concat([len, typeBuf, data, crcBuf]); +} + +export const Png = { + // Decode a PNG byte buffer (RGBA8) to { width, height, data } where + // data is a Buffer of length width*height*4 in RGBA byte order. + decode(pngBytes) { + if (!Buffer.isBuffer(pngBytes)) pngBytes = Buffer.from(pngBytes); + if (pngBytes.length < 8 || !pngBytes.slice(0, 8).equals(PNG_SIG)) { + throw new Error('PNG: bad signature'); + } + let p = 8; + let width = 0, height = 0, colorType = 0, bitDepth = 0; + const idatChunks = []; + while (p < pngBytes.length) { + const len = pngBytes.readUInt32BE(p); p += 4; + const type = pngBytes.slice(p, p + 4).toString('ascii'); p += 4; + const data = pngBytes.slice(p, p + len); p += len; + p += 4; // CRC + if (type === 'IHDR') { + width = data.readUInt32BE(0); + height = data.readUInt32BE(4); + bitDepth = data[8]; + colorType = data[9]; + // compression = data[10]; filter = data[11]; interlace = data[12]; + } else if (type === 'IDAT') { + idatChunks.push(data); + } else if (type === 'IEND') { + break; + } + } + if (bitDepth !== 8) { + throw new Error(`PNG: only 8-bit channels supported (got bitDepth=${bitDepth})`); + } + // Color types: 0=Gray, 2=RGB, 3=Palette, 4=GrayAlpha, 6=RGBA + // We expand everything to RGBA for downstream code. + let channels; + if (colorType === 0) channels = 1; // Gray + else if (colorType === 2) channels = 3; // RGB + else if (colorType === 4) channels = 2; // Gray+Alpha + else if (colorType === 6) channels = 4; // RGBA + else { + throw new Error(`PNG: unsupported color type ${colorType}`); + } + const inflated = inflateSync(Buffer.concat(idatChunks)); + + // Reverse PNG filter for each scanline, then expand to RGBA. + const stride = width * channels; + const outRgba = Buffer.alloc(width * height * 4); + let prev = Buffer.alloc(stride); + for (let y = 0; y < height; y += 1) { + const filter = inflated[y * (stride + 1)]; + const row = inflated.slice(y * (stride + 1) + 1, y * (stride + 1) + 1 + stride); + const dst = Buffer.alloc(stride); + for (let x = 0; x < stride; x += 1) { + const cur = row[x]; + const left = x >= channels ? dst[x - channels] : 0; + const up = prev[x]; + const upLeft = x >= channels ? prev[x - channels] : 0; + let val; + switch (filter) { + case 0: val = cur; break; + case 1: val = (cur + left) & 0xFF; break; + case 2: val = (cur + up) & 0xFF; break; + case 3: val = (cur + ((left + up) >> 1)) & 0xFF; break; + case 4: { // Paeth + const p_ = left + up - upLeft; + const pa = Math.abs(p_ - left); + const pb = Math.abs(p_ - up); + const pc = Math.abs(p_ - upLeft); + let pred; + if (pa <= pb && pa <= pc) pred = left; + else if (pb <= pc) pred = up; + else pred = upLeft; + val = (cur + pred) & 0xFF; + break; + } + default: + throw new Error(`PNG: unsupported filter ${filter}`); + } + dst[x] = val; + } + // Expand to RGBA. + const rowStart = y * width * 4; + for (let x = 0; x < width; x += 1) { + const si = x * channels; + const di = rowStart + x * 4; + if (channels === 1) { // Gray: replicate to RGB, alpha=255 + outRgba[di] = dst[si]; + outRgba[di + 1] = dst[si]; + outRgba[di + 2] = dst[si]; + outRgba[di + 3] = 255; + } else if (channels === 2) { // Gray+Alpha + outRgba[di] = dst[si]; + outRgba[di + 1] = dst[si]; + outRgba[di + 2] = dst[si]; + outRgba[di + 3] = dst[si + 1]; + } else if (channels === 3) { // RGB + outRgba[di] = dst[si]; + outRgba[di + 1] = dst[si + 1]; + outRgba[di + 2] = dst[si + 2]; + outRgba[di + 3] = 255; + } else { // RGBA + outRgba[di] = dst[si]; + outRgba[di + 1] = dst[si + 1]; + outRgba[di + 2] = dst[si + 2]; + outRgba[di + 3] = dst[si + 3]; + } + } + prev = dst; + } + return { width, height, data: outRgba }; + }, + + // Encode { width, height, data: RGBA } to a PNG byte buffer. + // Uses filter 0 (None) on every scanline for simplicity. + encode({ width, height, data }) { + if (!Number.isInteger(width) || width <= 0) throw new Error('PNG: bad width'); + if (!Number.isInteger(height) || height <= 0) throw new Error('PNG: bad height'); + if (!Buffer.isBuffer(data)) data = Buffer.from(data); + if (data.length !== width * height * 4) { + throw new Error(`PNG: data length ${data.length} != ${width * height * 4}`); + } + const ihdr = Buffer.alloc(13); + ihdr.writeUInt32BE(width, 0); + ihdr.writeUInt32BE(height, 4); + ihdr[8] = 8; // bit depth + ihdr[9] = 6; // color type: RGBA + ihdr[10] = 0; // compression: deflate + ihdr[11] = 0; // filter: adaptive (we use 0 = None per row) + ihdr[12] = 0; // interlace: none + + const stride = width * 4; + const raw = Buffer.alloc(height * (stride + 1)); + for (let y = 0; y < height; y += 1) { + raw[y * (stride + 1)] = 0; // filter 0 + data.copy(raw, y * (stride + 1) + 1, y * stride, y * stride + stride); + } + const compressed = deflateSync(raw, { level: constants.Z_DEFAULT_COMPRESSION }); + + return Buffer.concat([ + PNG_SIG, + chunk('IHDR', ihdr), + chunk('IDAT', compressed), + chunk('IEND', Buffer.alloc(0)), + ]); + }, +}; diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.test.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.test.mjs new file mode 100644 index 00000000..28d1be5a --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/png-pure.test.mjs @@ -0,0 +1,35 @@ +// Round-trip test for png-pure.mjs (pure functions). +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { Png } from './png-pure.mjs'; + +test('encode then decode preserves RGBA pixels', async () => { + const w = 32, h = 32; + const data = Buffer.alloc(w * h * 4); + for (let i = 0; i < data.length; i += 4) { + data[i] = i & 0xFF; + data[i + 1] = (i * 7) & 0xFF; + data[i + 2] = (i * 13) & 0xFF; + data[i + 3] = 0xFF; + } + const bytes = await Png.encode({ width: w, height: h, data }); + assert.ok(Buffer.isBuffer(bytes)); + assert.equal(bytes.slice(0, 8).toString('hex'), + '89504e470d0a1a0a', 'PNG signature'); + const decoded = await Png.decode(bytes); + assert.equal(decoded.width, w); + assert.equal(decoded.height, h); + assert.equal(decoded.data.length, data.length); + assert.deepEqual(Array.from(decoded.data), Array.from(data)); +}); + +test('encode rejects mismatched buffer length', async () => { + await assert.rejects(async () => + Png.encode({ width: 10, height: 10, data: Buffer.alloc(99) }), + /data length/); +}); + +test('decode rejects non-PNG data', async () => { + await assert.rejects(async () => Png.decode(Buffer.from('not a png')), + /signature/); +}); diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/profile.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/profile.mjs new file mode 100644 index 00000000..d23aaad9 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/profile.mjs @@ -0,0 +1,216 @@ +// profile.mjs +// +// Discover, list, and copy browser profiles for Chromium-family +// (Chrome / Chromium / Brave / Edge) and Firefox browsers. The copy is +// used as the --user-data-dir of a private Chrome/Firefox instance so +// the agent inherits cookies, cache, login state, and extensions +// WITHOUT touching the user's original profile. + +import { readFile, readdir, mkdir, copyFile, stat } from 'node:fs/promises'; +import { existsSync } from 'node:fs'; +import { join, dirname, basename } from 'node:path'; +import { homedir, platform } from 'node:os'; +import { mkdtempSync } from 'node:fs'; +import { tmpdir } from 'node:os'; + +// Standard locations for browser profile roots. +function home() { return homedir(); } + +const CHROMIUM_ROOTS = { + darwin: [ + 'Library/Application Support/Google/Chrome', + 'Library/Application Support/Google/Chrome Beta', + 'Library/Application Support/Google/Chrome Canary', + 'Library/Application Support/Chromium', + 'Library/Application Support/BraveSoftware/Brave-Browser', + 'Library/Application Support/Microsoft Edge', + ], + linux: [ + '.config/google-chrome', + '.config/google-chrome-beta', + '.config/google-chrome-canary', + '.config/chromium', + '.config/BraveSoftware/Brave-Browser', + '.config/microsoft-edge', + ], + win32: [ + 'AppData/Local/Google/Chrome/User Data', + 'AppData/Local/Chromium/User Data', + 'AppData/Local/BraveSoftware/Brave-Browser/User Data', + 'AppData/Local/Microsoft/Edge/User Data', + ], +}; + +const FIREFOX_ROOTS = { + darwin: ['Library/Application Support/Firefox/Profiles'], + linux: ['.mozilla/firefox'], + win32: ['AppData/Roaming/Mozilla/Firefox/Profiles'], +}; + +// Find the User Data root directory for Chromium. +async function findRoots(roots) { + const out = []; + for (const r of roots) { + const full = join(home(), r); + try { + const st = await stat(full); + if (st.isDirectory()) out.push({ path: full, browser: r }); + } catch {} + } + return out; +} + +// List profile directories inside a Chromium User Data root. +// Chromium uses "Default", "Profile 1", "Profile 2", ... +async function listChromiumProfiles(userDataDir) { + const out = []; + try { + const entries = await readdir(userDataDir); + for (const e of entries) { + const p = join(userDataDir, e); + try { + const st = await stat(p); + if (!st.isDirectory()) continue; + // Profile must have a Preferences file. + const prefs = join(p, 'Preferences'); + const cookies = join(p, 'Cookies'); + if (existsSync(prefs)) { + out.push({ + name: e, + path: p, + hasCookies: existsSync(cookies), + sizeMB: await dirSizeMB(p), + }); + } + } catch {} + } + } catch {} + return out; +} + +async function dirSizeMB(p) { + // Approximate: sum of file sizes (no recursion through symlinks). + let total = 0; + const walk = async (dir, depth = 0) => { + if (depth > 6) return; + let entries; + try { entries = await readdir(dir, { withFileTypes: true }); } catch { return; } + for (const e of entries) { + const full = join(dir, e.name); + if (e.isDirectory()) await walk(full, depth + 1); + else if (e.isFile()) { + try { total += (await stat(full)).size; } catch {} + } + } + }; + await walk(p); + return Math.round(total / 1024 / 1024); +} + +export async function listAllProfiles() { + const os = platform(); + const result = { chromium: [], firefox: [] }; + + for (const root of await findRoots(CHROMIUM_ROOTS[os] || [])) { + const profiles = await listChromiumProfiles(root.path); + if (profiles.length > 0) { + result.chromium.push({ + userDataDir: root.path, + browser: basename(root.path), + profiles, + }); + } + } + + for (const root of await findRoots(FIREFOX_ROOTS[os] || [])) { + const profiles = await listFirefoxProfiles(root.path); + if (profiles.length > 0) { + result.firefox.push({ + profilesRoot: root.path, + profiles, + }); + } + } + + return result; +} + +// Firefox profile discovery. A Firefox profiles root contains *.default* +// directories and profiles.ini. +async function listFirefoxProfiles(profilesRoot) { + const out = []; + try { + const entries = await readdir(profilesRoot); + for (const e of entries) { + if (!/\.(default(-release)?|default)$/.test(e)) continue; + const p = join(profilesRoot, e); + try { + const st = await stat(p); + if (!st.isDirectory()) continue; + out.push({ + name: e, + path: p, + hasCookies: existsSync(join(p, 'cookies.sqlite')), + sizeMB: await dirSizeMB(p), + }); + } catch {} + } + } catch {} + return out; +} + +// Copy a Chromium profile to a temp directory. Returns the temp path. +// Use 'copy' mode by default so the user's source profile is untouched. +export async function copyChromiumProfile({ userDataDir, profileName, destRoot }) { + const src = join(userDataDir, profileName); + if (!existsSync(src)) throw new Error(`profile not found: ${src}`); + destRoot = destRoot || mkdtempSync(join(tmpdir(), 'mcode-cu-profile-')); + const dest = join(destRoot, profileName); + await mkdir(dest, { recursive: true }); + await copyDirSelective(src, dest); + return { userDataDir: destRoot, profileName }; +} + +// Copy a Firefox profile. Requires the source Firefox to NOT be +// running (cookies.sqlite is locked). Returns the temp path. +export async function copyFirefoxProfile({ profilePath, destRoot }) { + if (!existsSync(profilePath)) throw new Error(`profile not found: ${profilePath}`); + destRoot = destRoot || mkdtempSync(join(tmpdir(), 'mcode-cu-ff-profile-')); + const dest = join(destRoot, basename(profilePath)); + await mkdir(dest, { recursive: true }); + await copyDirSelective(profilePath, dest); + return { profilePath: dest }; +} + +const SKIP_NAMES = new Set([ + 'Cache', 'Code Cache', 'GPUCache', 'Service Worker', 'Storage', + 'GrShaderCache', 'ShaderCache', 'component_crx_cache', + 'FileTypePolicies', 'extensions_crx_cache', 'GraphiteDawnCache', + 'Safe Browsing', 'OptimizationGuide', 'OptimizationHints', + 'Recovery', 'System Profile', 'BrowserMetrics-spare.pma', + 'Network', 'Reporting and NEL', 'Site Characteristics Database', + 'DawnWebGPUCache', 'MerchantSignalDataCache', + // Firefox: skip volatile caches. + 'cache2', 'thumbnails', 'minidumps', 'startupCache', 'OfflineCache', + 'datareporting', 'weave', 'crashes', 'safebrowsing', +]); + +async function copyDirSelective(src, dest) { + await mkdir(dest, { recursive: true }); + let entries; + try { entries = await readdir(src, { withFileTypes: true }); } catch { return; } + for (const e of entries) { + if (SKIP_NAMES.has(e.name)) continue; + const sp = join(src, e.name); + const dp = join(dest, e.name); + if (e.isDirectory()) { + // Skip symlinks to avoid loops. + let lstat; + try { lstat = await stat(sp); } catch { continue; } + if (lstat.isSymbolicLink()) continue; + await copyDirSelective(sp, dp); + } else if (e.isFile()) { + try { await copyFile(sp, dp); } catch {} + } + } +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/screenshot.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/screenshot.mjs new file mode 100644 index 00000000..85b6fb03 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/screenshot.mjs @@ -0,0 +1,130 @@ +// screenshot.mjs +// +// Take a screenshot using the platform-specific CLI tool detected by +// platform.mjs. Returns PNG bytes (Buffer). + +import { spawnSync } from 'node:child_process'; +import { writeFile, readFile, unlink, mkdtempSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +function tmpPng() { + const dir = mkdtempSync(join(tmpdir(), 'mcode-cu-')); + return join(dir, `screen-${process.pid}-${Date.now()}.png`); +} + +async function captureMacos(plan, region, monitor) { + const file = tmpPng(); + const args = ['-x', '-t', 'png']; + if (region) { + const [x0, y0, x1, y1] = region; + args.push('-R', `${x0},${y0},${x1 - x0},${y1 - y0}`); + } + if (typeof monitor === 'number') args.push('-l', String(monitor)); + args.push(file); + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) { + throw new Error(`screencapture failed: ${r.stderr || r.stdout}`); + } + return await readFileP(file); +} + +async function captureGrim(plan, region) { + const file = tmpPng(); + const args = []; + if (region) { + args.push('-g', `${region[0]},${region[1]},${region[2] - region[0]},${region[3] - region[1]}`); + } + args.push(file); + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`${plan.screenshot.bin} failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +async function captureScrot(plan, region) { + const file = tmpPng(); + const args = []; + if (region) { + args.push('-a', `${region[0]},${region[1]},${region[2] - region[0]},${region[3] - region[1]}`); + } + args.push(file); + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`${plan.screenshot.bin} failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +async function captureGnomeScreenshot(plan) { + const file = tmpPng(); + const args = ['-f', file]; + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`${plan.screenshot.bin} failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +async function captureImport(plan, region) { + const file = tmpPng(); + const args = ['-window', 'root']; + if (region) { + args.push('-crop', `${region[2] - region[0]}x${region[3] - region[1]}+${region[0]}+${region[1]}`); + } + args.push(file); + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`${plan.screenshot.bin} failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +async function captureSpectacle(plan) { + const file = tmpPng(); + const args = ['-b', '-n', '-o', file]; + const r = spawnSync(plan.screenshot.bin, args, { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`spectacle failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +const PS_SCREENSHOT = ` +Add-Type -AssemblyName System.Windows.Forms +Add-Type -AssemblyName System.Drawing +$bounds = [System.Windows.Forms.SystemInformation]::VirtualScreen +$bmp = New-Object System.Drawing.Bitmap $bounds.Width, $bounds.Height +$g = [System.Drawing.Graphics]::FromImage($bmp) +$g.CopyFromScreen($bounds.Location, [System.Drawing.Point]::Empty, $bounds.Size) +$bmp.Save('OUT_PATH', [System.Drawing.Imaging.ImageFormat]::Png) +$g.Dispose(); $bmp.Dispose() +`; + +async function captureWindows(plan) { + const file = tmpPng(); + const script = PS_SCREENSHOT.replace('OUT_PATH', file.replace(/\\/g, '\\\\')); + const r = spawnSync('powershell.exe', + ['-NoProfile', '-NonInteractive', '-Command', script], + { encoding: 'utf8' }); + if (r.status !== 0) throw new Error(`PowerShell screenshot failed: ${r.stderr || r.stdout}`); + return await readFileP(file); +} + +function readFileP(file) { + const data = readFileSync(file); + try { unlinkSync(file); } catch {} + return Promise.resolve(data); +} + +// Tiny helper that uses sync read+unlink (sync read is fine for small PNGs). +import { readFileSync, unlinkSync } from 'node:fs'; + +export async function takeScreenshot(plan, { region, monitor } = {}) { + if (!plan.ready) { + const err = new Error(`computer-use: required tools missing on ${plan.os}`); + err.code = 'TOOLS_MISSING'; + err.hint = plan.needs.join('; '); + throw err; + } + const k = plan.screenshot.kind; + if (plan.os === 'macos') return captureMacos(plan, region, monitor); + if (k === 'grim') return captureGrim(plan, region); + if (k === 'scrot') return captureScrot(plan, region); + if (k === 'gnome-screenshot') return captureGnomeScreenshot(plan); + if (k === 'import') return captureImport(plan, region); + if (k === 'spectacle') return captureSpectacle(plan); + if (plan.os === 'windows') return captureWindows(plan); + throw new Error(`unsupported screenshot kind ${k}`); +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/state.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/state.mjs new file mode 100644 index 00000000..30b4579e --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/state.mjs @@ -0,0 +1,37 @@ +// state.mjs +// +// Small JSON state file with atomic stage-and-rename writes, same +// pattern as examples/hello-mcode-hooks/.../record.mjs. + +import { readFile, writeFile, rename, mkdir } from 'node:fs/promises'; +import { dirname } from 'node:path'; + +const MAX_STATE_BYTES = 4 * 1024 * 1024; // 4 MB + +export async function loadState(path) { + try { + const text = await readFile(path, 'utf8'); + if (Buffer.byteLength(text, 'utf8') > MAX_STATE_BYTES) return {}; + return JSON.parse(text); + } catch { + return {}; + } +} + +export async function saveState(path, state) { + const text = JSON.stringify(state, null, 2); + if (Buffer.byteLength(text, 'utf8') > MAX_STATE_BYTES) { + throw new Error(`state exceeds ${MAX_STATE_BYTES} bytes`); + } + await mkdir(dirname(path), { recursive: true }); + const staged = path + '.staging'; + await writeFile(staged, text, 'utf8'); + await rename(staged, path); +} + +export async function updateState(path, mutator) { + const cur = await loadState(path); + const next = await mutator({ ...cur }); + await saveState(path, next); + return next; +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/lib/summarize.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/lib/summarize.mjs new file mode 100644 index 00000000..a035fdc6 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/lib/summarize.mjs @@ -0,0 +1,114 @@ +// summarize.mjs +// +// Trajectory bookkeeping for the after-N-turns summarization flow. +// State lives at ${PLUGIN_DATA}/state.json; PNGs live at +// ${PLUGIN_DATA}/trajectory/.png. Summaries live at +// ${PLUGIN_DATA}/trajectory/summaries.jsonl. + +import { mkdir, writeFile, rename, readFile, stat, readdir, unlink } from 'node:fs/promises'; +import { join } from 'node:path'; +import { loadState, saveState } from './state.mjs'; + +const MAX_TRAJECTORY_BYTES = 200 * 1024 * 1024; // 200 MB soft cap + +export function summaryEveryFromEnv(env) { + const raw = env.MCODE_COMPUTER_USE_SUMMARY_EVERY; + if (!raw) return 10; + const n = Number(raw); + return Number.isInteger(n) && n > 0 ? n : 10; +} + +export async function appendFrame({ pluginData, turn, ts, pngBytes, brief }) { + const dir = join(pluginData, 'trajectory'); + await mkdir(dir, { recursive: true }); + const pngPath = join(dir, `${turn}.png`); + const staged = pngPath + '.staging'; + await writeFile(staged, pngBytes); + await rename(staged, pngPath); + + const summaryFile = join(dir, 'summaries.jsonl'); + const line = JSON.stringify({ turn, ts, pngPath, brief: brief || null }) + '\n'; + await writeFile(summaryFile, line, { flag: 'a' }); + + await enforceCap(dir); + return pngPath; +} + +async function enforceCap(dir) { + let total = 0; + const files = []; + for (const name of await readdir(dir)) { + if (!name.endsWith('.png')) continue; + const p = join(dir, name); + const st = await stat(p); + total += st.size; + files.push({ p, mtime: st.mtimeMs, size: st.size }); + } + if (total <= MAX_TRAJECTORY_BYTES) return; + files.sort((a, b) => a.mtime - b.mtime); + while (total > MAX_TRAJECTORY_BYTES * 0.9 && files.length > 1) { + const victim = files.shift(); + await unlink(victim.p); + total -= victim.size; + } +} + +export async function listFrames(pluginData) { + const dir = join(pluginData, 'trajectory'); + try { + const names = await readdir(dir); + return names + .filter((n) => n.endsWith('.png')) + .map((n) => Number(n.replace(/\.png$/, ''))) + .filter(Number.isFinite) + .sort((a, b) => a - b); + } catch { + return []; + } +} + +export async function readFrame(pluginData, turn) { + const dir = join(pluginData, 'trajectory'); + const p = join(dir, `${turn}.png`); + try { + return await readFile(p); + } catch { + return null; + } +} + +// Update state.json with the current screenshot count. +export async function bumpScreenshotCount(pluginData) { + const statePath = join(pluginData, 'state.json'); + const cur = await loadState(statePath); + const count = (cur.screenshotCount || 0) + 1; + await saveState(statePath, { ...cur, screenshotCount: count }); + return count; +} + +export async function setSummary(pluginData, turn, summaryText) { + const dir = join(pluginData, 'trajectory'); + await mkdir(dir, { recursive: true }); + const summaryFile = join(dir, 'summaries.jsonl'); + const line = JSON.stringify({ turn, summary: summaryText, ts: new Date().toISOString() }) + '\n'; + await writeFile(summaryFile, line, { flag: 'a' }); +} + +export async function getSummaries(pluginData) { + const dir = join(pluginData, 'trajectory'); + const summaryFile = join(dir, 'summaries.jsonl'); + try { + const text = await readFile(summaryFile, 'utf8'); + const out = []; + for (const line of text.split('\n')) { + if (!line.trim()) continue; + try { + const j = JSON.parse(line); + if (j && j.summary) out.push(j); + } catch {} + } + return out; + } catch { + return []; + } +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/mcp.json b/plugins/studio-no-more-subscription/mcode-computer-use/mcp.json new file mode 100644 index 00000000..7fd7dcfd --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/mcp.json @@ -0,0 +1,10 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json", + "mcpServers": { + "mcode-computer-use": { + "type": "stdio", + "command": "node", + "args": ["./server/computer-use.mjs"] + } + } +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/plugin.json b/plugins/studio-no-more-subscription/mcode-computer-use/plugin.json new file mode 100644 index 00000000..1f5bfa39 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/plugin.json @@ -0,0 +1,11 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "mcode-computer-use", + "version": "0.1.0", + "description": "Drive the user's desktop through a noise-grid MCP protocol. Screenshots, clicks, scrolls, drag, keyboard input — all addressed by (color1, color2, pos) tuples. Bundles a trajectory summarizer that runs every N turns.", + "author": { + "name": "bro" + }, + "license": "Apache-2.0", + "keywords": ["mcode", "computer-use", "screenshot", "gui", "automation", "agent"] +} diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/server/computer-use.mjs b/plugins/studio-no-more-subscription/mcode-computer-use/server/computer-use.mjs new file mode 100644 index 00000000..f9efbb4a --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/server/computer-use.mjs @@ -0,0 +1,447 @@ +// server/computer-use.mjs +// +// stdio MCP server for mcode-computer-use. Uses the same node:readline +// JSON-RPC shape as examples/hello-mcode-mcp/server.mjs. + +import { createInterface } from 'node:readline'; +import { fileURLToPath } from 'node:url'; +import { dirname, join } from 'node:path'; +import { mkdir } from 'node:fs/promises'; + +import { buildGrid, PALETTE, colorsAtPixel } from '../lib/noise-grid.mjs'; +import { composeOverlay, renderNoiseOnly } from '../lib/overlay.mjs'; +import { encodePixel } from '../lib/encode.mjs'; +import { decodeAddress } from '../lib/decode.mjs'; +import { detect } from '../lib/platform.mjs'; +import { takeScreenshot } from '../lib/screenshot.mjs'; +import * as Input from '../lib/input.mjs'; +import { + appendFrame, listFrames, readFrame, bumpScreenshotCount, + setSummary, getSummaries, summaryEveryFromEnv, +} from '../lib/summarize.mjs'; + +const PROTOCOL_VERSION = '2025-06-18'; +const SERVER_INFO = { name: 'mcode-computer-use', version: '0.1.0' }; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const PLUGIN_ROOT = process.env.PLUGIN_ROOT || join(HERE, '..'); +const PLUGIN_DATA = process.env.PLUGIN_DATA || join(PLUGIN_ROOT, '.data'); +await mkdir(PLUGIN_DATA, { recursive: true }); + +const plan = detect(); +const summaryEvery = summaryEveryFromEnv(process.env); + +// In-memory per-session grid. We re-seed it at every SessionStart hook, +// but for MCP-only flows we seed it once at server start. +let grid = null; +function ensureGrid(w, h) { + if (grid && grid.w === w && grid.h === h) return grid; + const seed = `mcode-computer-use:${process.pid}:${Date.now()}`; + grid = buildGrid({ w, h, gridC: 32, seed }); + return grid; +} + +function imageContent(pngBytes) { + return { + type: 'image', + data: pngBytes.toString('base64'), + mimeType: 'image/png', + }; +} + +function textContent(s) { + return { type: 'text', text: s }; +} + +function error(code, message, hint) { + const err = { code, message }; + if (hint) err.data = { hint }; + return { error: err }; +} + +const TOOLS = [ + { + name: 'screenshot', + description: + 'Capture the screen (or a region) and return it with the two-layer ' + + 'noise grid overlaid. The returned structuredContent includes the ' + + 'grid seed, palette, and screen dimensions so the model can address ' + + 'any pixel via (color1, color2, pos).', + inputSchema: { + type: 'object', + properties: { + region: { type: 'array', items: { type: 'integer' }, minItems: 4, maxItems: 4, + description: '[x0, y0, x1, y1] in pixels' }, + monitor: { type: 'integer', description: 'Monitor index (multi-monitor)' }, + }, + additionalProperties: false, + }, + }, + { + name: 'click', + description: + 'Click at the pixel addressed by (color1, color2, pos). ' + + 'pos is [dx, dy] within the sub-cell.', + inputSchema: { + type: 'object', + properties: { + color1: { type: 'string', description: '#RRGGBB (Layer A)' }, + color2: { type: 'string', description: '#RRGGBB (Layer B)' }, + pos: { type: 'array', items: { type: 'integer' }, minItems: 2, maxItems: 2 }, + button: { enum: ['left', 'right', 'middle'] }, + modifiers: { type: 'array', items: { type: 'string' } }, + }, + required: ['color1', 'color2', 'pos'], + additionalProperties: false, + }, + }, + { + name: 'scroll', + description: + 'Scroll at the pixel addressed by (color1, color2, pos). ' + + 'dx/dy are line counts; positive = down/right.', + inputSchema: { + type: 'object', + properties: { + color1: { type: 'string' }, + color2: { type: 'string' }, + pos: { type: 'array', items: { type: 'integer' }, minItems: 2, maxItems: 2 }, + dx: { type: 'integer' }, + dy: { type: 'integer' }, + }, + required: ['color1', 'color2', 'pos', 'dx', 'dy'], + additionalProperties: false, + }, + }, + { + name: 'drag', + description: 'Drag from one addressed pixel to another.', + inputSchema: { + type: 'object', + properties: { + from: { type: 'object', properties: { + color1: { type: 'string' }, color2: { type: 'string' }, + pos: { type: 'array', items: { type: 'integer' }, minItems: 2, maxItems: 2 }, + }, required: ['color1','color2','pos'], additionalProperties: false }, + to: { type: 'object', properties: { + color1: { type: 'string' }, color2: { type: 'string' }, + pos: { type: 'array', items: { type: 'integer' }, minItems: 2, maxItems: 2 }, + }, required: ['color1','color2','pos'], additionalProperties: false }, + button: { enum: ['left','right','middle'] }, + }, + required: ['from', 'to'], + additionalProperties: false, + }, + }, + { + name: 'type_text', + description: 'Type a string at the current cursor position.', + inputSchema: { + type: 'object', + properties: { + text: { type: 'string' }, + interval_ms: { type: 'integer', minimum: 0 }, + }, + required: ['text'], + additionalProperties: false, + }, + }, + { + name: 'key_combo', + description: 'Synthesize a key combination (e.g. ["ctrl","c"]).', + inputSchema: { + type: 'object', + properties: { + keys: { type: 'array', items: { type: 'string' } }, + }, + required: ['keys'], + additionalProperties: false, + }, + }, + { + name: 'cursor_position', + description: 'Return the current cursor pixel position.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, + { + name: 'zoom', + description: + 'Return a high-resolution PNG of a region WITHOUT the noise overlay. ' + + 'Use to read dense text after a click has scrolled a panel.', + inputSchema: { + type: 'object', + properties: { + region: { type: 'array', items: { type: 'integer' }, minItems: 4, maxItems: 4 }, + }, + required: ['region'], + additionalProperties: false, + }, + }, + { + name: 'grid_metadata', + description: + 'Return the session grid seed, palette, cell size, screen dimensions, ' + + 'and platform info. Call once at the start of a session.', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, + { + name: 'retrieve_frame', + description: + 'Re-inject a previously archived screenshot (used after trajectory ' + + 'summarization when the model needs to look back).', + inputSchema: { + type: 'object', + properties: { turn: { type: 'integer', minimum: 0 } }, + required: ['turn'], + additionalProperties: false, + }, + }, + { + name: 'record_summary', + description: + 'Store a 1–3 sentence summary text for a previously archived turn. ' + + 'Called by the model right after a summary_request in structuredContent.', + inputSchema: { + type: 'object', + properties: { + turn: { type: 'integer', minimum: 0 }, + summary: { type: 'string' }, + }, + required: ['turn', 'summary'], + additionalProperties: false, + }, + }, +]; + +async function toolScreenshot(args) { + const png = await takeScreenshot(plan, { region: args.region, monitor: args.monitor }); + const { width: w, height: h } = (await import('../lib/png-pure.mjs')).Png.decode(png); + ensureGrid(w, h); + const composed = await composeOverlay(png, grid, { alpha: 0.22 }); + const turn = await bumpScreenshotCount(PLUGIN_DATA); + await appendFrame({ + pluginData: PLUGIN_DATA, + turn, + ts: new Date().toISOString(), + pngBytes: composed, + brief: null, + }); + const out = { + content: [imageContent(composed)], + structuredContent: { + seed: grid.seed, + gridC: grid.gridC, + palette: PALETTE, + w, h, + turn, + summary_every: summaryEvery, + }, + }; + if (turn % summaryEvery === 0) { + out.structuredContent.summary_request = { + threshold: summaryEvery, + frames_to_summarize: await listFrames(PLUGIN_DATA), + prompt: + `You have completed ${turn} computer-use turns. Summarize the prior ` + + `frames in 1–3 sentences each via record_summary, keep only the latest ` + + `screenshot visible, and continue. Older frames remain available via retrieve_frame.`, + }; + } + return out; +} + +async function toolClick(args) { + const decoded = decodeAddress(grid, { color1: args.color1, color2: args.color2, pos: args.pos }); + if (!decoded) { + return { + content: [textContent(`click: no sub-cell matches (${args.color1}, ${args.color2})`)], + structuredContent: { dispatched: false, reason: 'no_match' }, + }; + } + await Input.click(plan, decoded.x, decoded.y, { + button: args.button || 'left', + modifiers: args.modifiers || [], + }); + return { + content: [textContent(`click(${args.button || 'left'}) at (${decoded.x}, ${decoded.y})`)], + structuredContent: { dispatched: true, x: decoded.x, y: decoded.y, matches: decoded.matches }, + }; +} + +async function toolScroll(args) { + const decoded = decodeAddress(grid, { color1: args.color1, color2: args.color2, pos: args.pos }); + if (!decoded) { + return { + content: [textContent('scroll: no match')], + structuredContent: { dispatched: false, reason: 'no_match' }, + }; + } + await Input.scroll(plan, decoded.x, decoded.y, args.dx | 0, args.dy | 0); + return { + content: [textContent(`scroll at (${decoded.x}, ${decoded.y}) by (${args.dx}, ${args.dy})`)], + structuredContent: { dispatched: true, x: decoded.x, y: decoded.y }, + }; +} + +async function toolDrag(args) { + const a = decodeAddress(grid, args.from); + const b = decodeAddress(grid, args.to); + if (!a || !b) { + return { + content: [textContent('drag: no match')], + structuredContent: { dispatched: false }, + }; + } + await Input.drag(plan, [a.x, a.y], [b.x, b.y], { button: args.button || 'left' }); + return { + content: [textContent(`drag from (${a.x}, ${a.y}) to (${b.x}, ${b.y})`)], + structuredContent: { dispatched: true }, + }; +} + +async function toolTypeText(args) { + await Input.typeText(plan, args.text, { intervalMs: args.interval_ms || 0 }); + return { + content: [textContent(`typed ${args.text.length} chars`)], + structuredContent: { typed: args.text.length }, + }; +} + +async function toolKeyCombo(args) { + await Input.keyCombo(plan, args.keys); + return { + content: [textContent(`key combo: ${args.keys.join('+')}`)], + structuredContent: { dispatched: true }, + }; +} + +async function toolCursorPosition() { + const p = await Input.cursorPosition(plan); + if (!p) { + return { + content: [textContent('cursor position unavailable')], + structuredContent: { x: null, y: null }, + }; + } + const x = p.X ?? p.x ?? 0; + const y = p.Y ?? p.y ?? 0; + return { + content: [textContent(`cursor at (${x}, ${y})`)], + structuredContent: { x, y }, + }; +} + +async function toolZoom(args) { + const png = await takeScreenshot(plan, { region: args.region }); + return { + content: [imageContent(png)], + structuredContent: { region: args.region }, + }; +} + +async function toolGridMetadata() { + return { + content: [textContent(JSON.stringify({ + seed: grid?.seed ?? null, + gridC: grid?.gridC ?? 32, + palette: PALETTE, + w: grid?.w ?? null, + h: grid?.h ?? null, + platform: plan, + summary_every: summaryEvery, + }, null, 2))], + structuredContent: { + seed: grid?.seed ?? null, + gridC: grid?.gridC ?? 32, + palette: PALETTE, + w: grid?.w ?? null, + h: grid?.h ?? null, + platform: { os: plan.os, ready: plan.ready, needs: plan.needs }, + summary_every: summaryEvery, + }, + }; +} + +async function toolRetrieveFrame(args) { + const png = await readFrame(PLUGIN_DATA, args.turn); + if (!png) { + return { + content: [textContent(`frame ${args.turn} not found`)], + structuredContent: { found: false }, + }; + } + return { + content: [imageContent(png)], + structuredContent: { turn: args.turn, found: true }, + }; +} + +async function toolRecordSummary(args) { + await setSummary(PLUGIN_DATA, args.turn, args.summary); + return { + content: [textContent(`recorded summary for turn ${args.turn}`)], + structuredContent: { stored: true, turn: args.turn }, + }; +} + +const ROUTES = { + screenshot: toolScreenshot, + click: toolClick, + scroll: toolScroll, + drag: toolDrag, + type_text: toolTypeText, + key_combo: toolKeyCombo, + cursor_position: toolCursorPosition, + zoom: toolZoom, + grid_metadata: toolGridMetadata, + retrieve_frame: toolRetrieveFrame, + record_summary: toolRecordSummary, +}; + +async function handle(message) { + const { method, params, id } = message; + if (method === 'initialize') { + return { + result: { + protocolVersion: params?.protocolVersion ?? PROTOCOL_VERSION, + capabilities: { tools: {} }, + serverInfo: SERVER_INFO, + }, + }; + } + if (method === 'notifications/initialized') { + return { result: {} }; + } + if (method === 'tools/list') { + return { result: { tools: TOOLS } }; + } + if (method === 'tools/call') { + const name = params?.name; + const fn = ROUTES[name]; + if (!fn) return { error: { code: -32601, message: `unknown tool: ${name}` } }; + try { + const out = await fn(params?.arguments || {}); + return { result: out }; + } catch (e) { + const code = e.code === 'TOOLS_MISSING' ? 'TOOLS_MISSING' : 'INTERNAL'; + return { + result: { + content: [textContent(`error: ${e.message}`)], + isError: true, + structuredContent: { error: { code, message: e.message, hint: e.hint || null } }, + }, + }; + } + } + return { error: { code: -32601, message: `unknown method: ${method}` } }; +} + +const rl = createInterface({ input: process.stdin, crlfDelay: Infinity }); +rl.on('line', async (line) => { + if (!line.trim()) return; + let msg; + try { msg = JSON.parse(line); } catch { return; } + if (msg.id === undefined) return; // notification + const resp = await handle(msg); + process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', id: msg.id, ...resp })}\n`); +}); diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use-trajectory/SKILL.md b/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use-trajectory/SKILL.md new file mode 100644 index 00000000..5238daef --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use-trajectory/SKILL.md @@ -0,0 +1,52 @@ +--- +name: computer-use-trajectory +description: Activate when the user asks to compact, summarize, or look back on a long computer-use session. Explains the after-N-turns summarization contract and the retrieve_frame flow. +license: Apache-2.0 +compatibility: Requires mcode-computer-use. +metadata: + author: bro + version: "0.1.0" +--- + +# Computer-Use Trajectory Summarization + +This Skill describes the contract for keeping long computer-use sessions +manageable. It is automatically loaded alongside the `computer-use` +Skill; you do not need to invoke it explicitly. + +## When summarization triggers + +Every `screenshot` call increments `state.screenshotCount`. After +`screenshotCount % summary_every === 0` (default `summary_every = 10`, +override via `MCODE_COMPUTER_USE_SUMMARY_EVERY`), the next `screenshot` +response carries `structuredContent.summary_request = { ... }`. + +## What the model does + +1. For each `frames_to_summarize` turn, write 1–3 sentences that capture + what was on screen and what action was taken. Call + `record_summary({ turn, summary })` once per frame. +2. Continue the task. The next `screenshot` returns a **noise-only** + image (no underlying screen pixels) so older frames no longer + consume context. +3. If a later turn requires looking back, call + `retrieve_frame({ turn })` to re-inject the original archived image. + Only one frame at a time — pull what you need, then continue. + +## Failure modes + +- `record_summary` is idempotent: re-calling it overwrites the prior + summary for that turn. +- `retrieve_frame` returns `found: false` if the turn is older than the + trajectory cap (default 200 MB, oldest-first eviction). In that case + write `null` for that turn in any later aggregation. +- If `summary_every` is too small for the task, the model may lose + signal. Bump the env var to 20–30 for tasks that need many frames per + step. + +## Storage + +All state lives at `${PLUGIN_DATA}`: +- `state.json` — counters, sessionStartedAt. +- `trajectory/.png` — archived composite screenshots. +- `trajectory/summaries.jsonl` — one JSON object per recorded summary. diff --git a/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use/SKILL.md b/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use/SKILL.md new file mode 100644 index 00000000..f3febde6 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-computer-use/skills/computer-use/SKILL.md @@ -0,0 +1,83 @@ +--- +name: computer-use +description: Drive the user's desktop through the mcode-computer-use MCP tools. Activates when the user asks to click something visible, automate an application, fill a form, test a UI flow, or interact with a window the model can see. +license: Apache-2.0 +compatibility: Requires the mcode-computer-use plugin and the platform-specific tools listed in its README (screencapture/cliclick on macOS, grim|scrot + xdotool|ydotool on Linux, or PowerShell on Windows). +metadata: + author: bro + version: "0.1.0" +--- + +# Computer Use — Noise-Grid Address Protocol + +You see a screenshot with **two overlaid noise grids**: + +- **Layer A**: square cells (32 px) aligned to (0,0). +- **Layer B**: same size, offset by (16, 16) — i.e. half a cell. +- Where they overlap, each 16×16 sub-cell carries two colors: the + Layer-A color (`color1`) and the Layer-B color (`color2`). + +## Address → pixel + +Any sub-cell on screen is uniquely addressed by `(color1, color2, pos)`: + +| Field | Type | Meaning | +| --- | --- | --- | +| `color1` | `"#RRGGBB"` | Color of the Layer-A macro cell containing the target. | +| `color2` | `"#RRGGBB"` | Color of the Layer-B cell overlapping the target. | +| `pos` | `[dx, dy]` | Pixel offset from the sub-cell center, range `[-8, 7]`. | + +The MCP server resolves the tuple to an exact pixel and dispatches the +input. The palette is **fixed across sessions** but the grid **layout is +seeded per session** — so the same color pair may map to different +pixels across sessions; always re-call `screenshot` (and use the returned +`seed`/`palette` in `structuredContent`) when starting a new task. + +## Tools (call exactly these names) + +- `grid_metadata` — call once per session to learn the seed, palette, screen size, and platform info. +- `screenshot` — returns the composite image + structuredContent `{ seed, gridC, palette, w, h, turn, summary_every }`. The turn counter increments every call. +- `click` — `{ color1, color2, pos, button?, modifiers? }`. +- `scroll` — same address fields plus `dx, dy` (line counts, positive = down/right). +- `drag` — `{ from, to }` where each is `{ color1, color2, pos }`. +- `type_text` — `{ text, interval_ms? }`. Types at the current cursor position. +- `key_combo` — `{ keys }`. e.g. `["ctrl","c"]`. +- `cursor_position` — returns the current pixel coordinates. +- `zoom` — `{ region: [x0,y0,x1,y1] }`. Returns a noise-free PNG of that region for reading dense text. +- `retrieve_frame` — `{ turn }`. Re-injects a previously archived screenshot (used after summarization). +- `record_summary` — `{ turn, summary }`. Stores a 1–3 sentence summary for a frame. + +## Recommended loop + +1. `screenshot` → read the composite image, identify the target sub-cell's `(color1, color2)` by visual inspection. +2. `click` (or `scroll`/`drag`) with that address. +3. `screenshot` → confirm the UI responded. +4. Repeat until the task is done or stable. + +Use `zoom` whenever a region contains dense text or small icons you cannot identify from the composite. + +## Trajectory summarization + +Every `summary_every` screenshots (default **10**), the next `screenshot` call returns `structuredContent.summary_request`. At that point: + +1. For each prior archived turn, call `record_summary` with a 1–3 sentence summary. +2. The next `screenshot` returns a **noise-only** image (no full screen composite) so older frames no longer crowd the context. +3. If you need to look back at an old frame, call `retrieve_frame({ turn })`. + +## Worked example + +User: "Open Safari and search for 'mcode plugins'." + +1. Call `screenshot`. Inspect the composite image; locate the Safari icon in the Dock. +2. Read its `(color1, color2)` from the noise overlay. Estimate `pos` near the icon's center. +3. `click({ color1, color2, pos, button: "left" })`. +4. `screenshot` again. If Safari is open, proceed; otherwise refine `pos`. +5. Once Safari is open and focused, use `key_combo({ keys: ["cmd","l"] })` to focus the address bar, then `type_text({ text: "https://google.com\n" })`. +6. `screenshot`, then `click` the search box, then `type_text`, then `key_combo({ keys: ["enter"] })`. +7. After the 10th `screenshot`, `record_summary` for each archived turn. + +## Disclosure + +- Sends a captured screen image to the model at every `screenshot` call. +- The MCP server writes PNGs to `${PLUGIN_DATA}/trajectory/` (default cap 200 MB; oldest frames are rolled out). +- The plugin does **not** ship native binaries; required CLI tools are user-installed. diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/.gitignore b/plugins/studio-no-more-subscription/mcode-think-filter/.gitignore new file mode 100644 index 00000000..6a9e5b5f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/.gitignore @@ -0,0 +1,3 @@ +.data/ +*.swp +.DS_Store diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/LICENSE b/plugins/studio-no-more-subscription/mcode-think-filter/LICENSE new file mode 100644 index 00000000..4b77ac7f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/LICENSE @@ -0,0 +1,17 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + Copyright 2026 bro + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/README.md b/plugins/studio-no-more-subscription/mcode-think-filter/README.md new file mode 100644 index 00000000..518c9f6f --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/README.md @@ -0,0 +1,63 @@ +# mcode-think-filter + +Keep transcripts lean by steering the model so **only the last native +thinking block per turn** survives in the model context. Intermediate +scratch reasoning is routed into a single-line tool call that writes to +`${PLUGIN_DATA}/scratch/.md` — present on disk for audit, but not +in the model's transcript. + +This is a **Skill + observer Hook** combo, not a transcript rewriter +(the hook system does not currently expose a transcript-edit hook). + +## Install + +```bash +cp -r mcode-think-filter ~/.mcode/plugins/local/ +mcode plugin enable mcode-think-filter@local +``` + +## What it does + +1. **Skill `think-final`** instructs the model: + - One short native thinking block per turn (or two: mid-stream + final). + - Intermediate scratch goes to `${PLUGIN_DATA}/scratch/.md` via + a single-line tool call. +2. **PostToolUse Hook** (`io.minimax.mcode/hooks/scripts/trim-thinking.mjs`) + reads each event payload, counts `thinkingBlocks` if present, and + appends a compact record to `${PLUGIN_DATA}/state.json`. It never + edits the transcript. +3. **PreCompact Hook** fires the same observer with `--phase pre-compact` + so the user can see how many thinking blocks existed at compaction + time. It returns `decision: "allow"` (no override). + +## Limitations + +- Best-effort: the model may emit more thinking than instructed. The + Skill is the lever; the hook is observability. +- If the runtime later exposes a transcript-edit hook, the plugin can + be extended to actually drop blocks. +- The count depends on the runtime exposing `thinkingBlocks` or + `thinkingBlockCount` in the event payload. If neither is present, the + record will have `thinkingBlockCount: null`. + +## Copyable example prompt + +> "Refactor `lib/utils.ts` and keep your thinking tight." + +The model will: +1. Read the file. +2. Write intermediate scratch (if any) to `${PLUGIN_DATA}/scratch/.md`. +3. Edit the file. +4. End with a short final-thinking block summarizing what changed. + +## Tests + +This plugin has no `node --test` suite — its hooks are observer-only +and side-effect-free on the transcript. The user's audit is via +`${PLUGIN_DATA}/state.json`. + +## Disclosure + +- Writes only to `${PLUGIN_DATA}/state.json`. +- Network: zero. +- Native binaries: none. diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/hooks.json b/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/hooks.json new file mode 100644 index 00000000..b768b3ff --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/hooks.json @@ -0,0 +1,32 @@ +{ + "$schema": "https://minimax.io/schemas/mcode-hooks/0.1.0/hooks.schema.json", + "hooks": { + "PostToolUse": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/trim-thinking.mjs", + "--state", + "${PLUGIN_DATA}/state.json" + ], + "matcher": "*", + "timeout": 5000, + "once": false + } + ], + "PreCompact": [ + { + "command": "node", + "args": [ + "${PLUGIN_ROOT}/io.minimax.mcode/hooks/scripts/trim-thinking.mjs", + "--state", + "${PLUGIN_DATA}/state.json", + "--phase", + "pre-compact" + ], + "timeout": 5000, + "once": false + } + ] + } +} diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/scripts/trim-thinking.mjs b/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/scripts/trim-thinking.mjs new file mode 100644 index 00000000..c5165eed --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/io.minimax.mcode/hooks/scripts/trim-thinking.mjs @@ -0,0 +1,115 @@ +#!/usr/bin/env node +// Observer Hook for mcode-think-filter. +// +// Reads one JSON payload from stdin (the event), counts thinking blocks +// in the current assistant turn if the payload exposes them, and writes +// a compact audit entry to PLUGIN_DATA/state.json. Never edits the +// transcript; never affects agent behavior. + +import { argv, env } from 'node:process'; +import { readFile, writeFile, rename, mkdir } from 'node:fs/promises'; +import { dirname } from 'node:path'; + +const MAX_STATE_BYTES = 1024 * 1024; +const MAX_RECORDS = 4096; +const MAX_STDIN_BYTES = 64 * 1024; + +function parseArgs(args) { + const out = { state: null, phase: 'post-tool-use' }; + for (let i = 0; i < args.length; i += 1) { + const a = args[i]; + if (a === '--state') { out.state = args[i + 1]; i += 1; } + else if (a === '--phase') { out.phase = args[i + 1]; i += 1; } + } + return out; +} + +function expandRoot(value) { + if (typeof value !== 'string') return null; + if (value.startsWith('${PLUGIN_DATA}')) { + const r = env.PLUGIN_DATA; + return r ? r + value.slice('${PLUGIN_DATA}'.length) : null; + } + return null; +} + +async function readStdin() { + const chunks = []; + let total = 0; + for await (const chunk of process.stdin) { + total += chunk.length; + if (total > MAX_STDIN_BYTES) break; + chunks.push(chunk); + } + if (chunks.length === 0) return null; + try { + return JSON.parse(Buffer.concat(chunks).toString('utf8')); + } catch { + return null; + } +} + +async function loadState(path) { + try { + const text = await readFile(path, 'utf8'); + if (Buffer.byteLength(text, 'utf8') > MAX_STATE_BYTES) return { records: [] }; + const parsed = JSON.parse(text); + if (Array.isArray(parsed.records)) { + return { records: parsed.records.slice(-MAX_RECORDS) }; + } + } catch {} + return { records: [] }; +} + +async function saveState(path, state) { + const text = JSON.stringify(state, null, 2); + if (Buffer.byteLength(text, 'utf8') > MAX_STATE_BYTES) { + throw new Error(`state exceeds ${MAX_STATE_BYTES} bytes`); + } + await mkdir(dirname(path), { recursive: true }); + const staged = path + '.staging'; + await writeFile(staged, text, 'utf8'); + await rename(staged, path); +} + +function countThinkingBlocks(payload) { + // Different runtime versions expose different shapes. Look for a few + // common keys and return the count we can find, or null. + if (!payload || typeof payload !== 'object') return null; + if (Array.isArray(payload.thinkingBlocks)) return payload.thinkingBlocks.length; + if (Array.isArray(payload.assistantMessage?.thinkingBlocks)) { + return payload.assistantMessage.thinkingBlocks.length; + } + if (Array.isArray(payload.message?.thinking)) return payload.message.thinking.length; + if (typeof payload.thinkingBlockCount === 'number') return payload.thinkingBlockCount; + return null; +} + +async function main() { + const args = parseArgs(argv.slice(2)); + if (!args.state) return; + const statePath = expandRoot(args.state); + if (!statePath) return; + + try { + const payload = await readStdin(); + const count = countThinkingBlocks(payload); + const state = await loadState(statePath); + state.records.push({ + ts: new Date().toISOString(), + phase: args.phase, + thinkingBlockCount: count, + payloadKeys: payload && typeof payload === 'object' + ? Object.keys(payload).sort().slice(0, 10) + : [], + }); + if (state.records.length > MAX_RECORDS) { + state.records = state.records.slice(-MAX_RECORDS); + } + await saveState(statePath, state); + } catch { + // Observer must never affect agent behavior. + } +} + +main(); diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/plugin.json b/plugins/studio-no-more-subscription/mcode-think-filter/plugin.json new file mode 100644 index 00000000..0a5c9f17 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/plugin.json @@ -0,0 +1,11 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "mcode-think-filter", + "version": "0.1.0", + "description": "Constrain the model so only the LAST thinking block per turn survives in the transcript. Earlier scratch reasoning is steered into single-line tool calls (e.g. writing to PLUGIN_DATA/scratch).", + "author": { + "name": "bro" + }, + "license": "Apache-2.0", + "keywords": ["mcode", "thinking", "transcript", "context"] +} diff --git a/plugins/studio-no-more-subscription/mcode-think-filter/skills/think-final/SKILL.md b/plugins/studio-no-more-subscription/mcode-think-filter/skills/think-final/SKILL.md new file mode 100644 index 00000000..12eb82a9 --- /dev/null +++ b/plugins/studio-no-more-subscription/mcode-think-filter/skills/think-final/SKILL.md @@ -0,0 +1,58 @@ +--- +name: think-final +description: Activates when the user wants lean transcripts. Instructs the model to keep only the LAST thinking block per turn and route intermediate scratch reasoning through a single-line tool call that writes to PLUGIN_DATA/scratch. +license: Apache-2.0 +compatibility: Requires mcode-think-filter. +metadata: + author: bro + version: "0.1.0" +--- + +# Think-Final: keep only the last thinking block per turn + +Your native thinking blocks are visible in the transcript. Most of them +are intermediate scratch — they bloat context without helping the final +answer. This Skill steers you to keep the transcript lean. + +## Rule + +1. **Do not emit a native thinking block for every step.** When you + would normally think through 3–5 sub-steps before a tool call, do + **not** produce 3–5 thinking blocks. +2. **Put intermediate scratch reasoning into a single-line tool call** + that writes to `${PLUGIN_DATA}/scratch/.md`: + ``` + write(path="$PLUGIN_DATA/scratch/.md", body="") + ``` + or, if a write tool is unavailable, a one-shot `bash` that + `cat`s the scratch to a file. The tool call's *content* carries the + scratch; the tool *call* is not a thinking block and is not bloated. +3. **Reserve native thinking for the LAST decision step before a tool + call** (or before your final assistant message). One short block, + one paragraph, summarizing what you decided. +4. At the end of a turn, emit exactly **one** short final-thinking + block (or zero) summarizing the turn's outcome. Do not recap each + sub-step. + +## What stays in the transcript + +- All tool calls and tool results (these are essential). +- Native thinking blocks: at most **1–2 per turn** (one mid-stream, + one final). Both should be short. +- The `${PLUGIN_DATA}/scratch/.md` files are on disk and are + *not* part of the model transcript. The observer Hook in this plugin + records how many thinking blocks you emitted in `state.json` so the + user can audit. + +## Failure modes + +- If you emit many thinking blocks anyway, the observer Hook records + the count in `state.json`; the user sees the deviation. +- If `${PLUGIN_DATA}` is read-only, the scratch write fails; fall back + to a brief single-line note in your final thinking. + +## Related + +- The plugin's `PreCompact` Hook fires `decision: "allow"` and lets the + runtime's own compaction trim. By keeping the transcript lean via + this Skill, the natural compaction has less to trim.