Skip to content

DEVELOPMENT.md

Latest commit

 

History

History
775 lines (605 loc) · 57.4 KB

DEVELOPMENT.md

File metadata and controls

775 lines (605 loc) · 57.4 KB
Raw

🛠️ EMRG — Development & Advanced Topics

This is the developer/advanced companion to README.md. It covers source installs, advanced configuration, architecture, and the full test/CI workflow — details intentionally kept out of the concise top-level README.

📦 Source Install (without the packaged installer)

The packaged installers (pkg / exe / AppImage) are recommended for end users — zero prerequisites, 100% offline. Prefer building from source? Use install.sh below.

🍎 macOS (source install)

Install:

curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash

Uninstall:

curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash -s -- purge

🐧 Linux (source install)

Install:

curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash

Uninstall:

curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash -s -- purge

🪟 Windows (WSL2, source install)

Install:

# Install WSL2 (skip if already installed)
wsl --install

# Enter WSL, then install
wsl
curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash

Uninstall:

# Run inside WSL
curl -sSL https://raw.githubusercontent.com/argszero/emrg/master/install.sh | bash -s -- purge

Source-install prerequisites (install.sh auto-detects and prompts): git, python 3.11+, uv. gh CLI recommended. For native Windows (non-WSL), use the packaged installer.

🔧 Advanced Configuration

The GUI rewrites config on save and drops comments — advanced users can edit ~/.emrg/config.toml directly (no manual editing needed for first-time setup; the GUI wizard handles it).

~/.emrg/config.toml template example (the GUI generates equivalent content on save):

[llm]
base_url = "https://api.deepseek.com"
api_key = "sk-..."
model = "deepseek-chat"
max_tokens = 8192
temperature = 0.7
context_window = 131072
auto_compact_threshold = 0.0
# vision: whether the model supports the OpenAI vision API (image_url). Keep false for
# non-vision models (e.g. DeepSeek) — pasted images degrade to text placeholders to avoid API errors.
# This is the DEFAULT: a [[llm.models]] entry's own `vision` key wins over it, and an
# entry without one (or no entry at all) falls back to this value — it never inherits
# the previous model's answer.
vision = false

# Multi-model support — use /model to switch between models
[[llm.models]]
name = "deepseek-v3"
model = "deepseek-chat"
context_window = 131072
vision = false

[[llm.models]]
name = "gpt-4o"
model = "gpt-4o"
context_window = 128000
vision = true

# The TUI's terminal theme. `auto` (the default) detects the terminal background and follows it;
# set `light` or `dark` to override a detection that guesses wrong.
[client]
theme = "auto"

Live reload — no restart needed. The daemon watches ~/.emrg/config.toml: every couple of seconds it reads the file and hashes it, parses it only when the hash moved, and applies the edit in place to the running process. Every key in [llm] moves for subsequent requests:

  • The tick reads the bytes rather than comparing (mtime, size). That pair was the first design and the windows-2025 leg falsified it: two writes milliseconds apart share a timestamp there, so an edit that also keeps the file's size leaves the comparison identical and the edit is silently never applied — the very complaint this feature removes. The cost is one small read plus a sha256 per tick, paid on every platform (coarse timestamps are a property of the mount, not of the OS) — worth knowing if ~/.emrg lives on a network filesystem.

  • model travels the same path /model uses — the usage anchors are invalidated and context_window re-resolved from the matching [[llm.models]] entry — and every connected client is told (model_set).

  • vision is resolved, and the resolved value is what is reported. It comes from one place — the matching entry's own key, else the top-level [llm] vision — and both the model_set frame and the ping/pong status frame carry the value the daemon will act on, so a client shows the effective capability rather than this file's declaration (the TUI prints it beside the model name in the status bar: [gpt-4o img] / [deepseek-chat no-img]). The declaration and the effective value are different things the moment a switch happens, which is why the file alone cannot answer "can I paste an image?".

  • A [[llm.models]] entry is matched by its name or its model, so /model <name> and /model <api-id> name the same entry — one matcher (config.find_model_entry) answers every lookup a switch makes, so the entry's context_window, its API id and its vision can never come from three different rows. Before that was one function, the context_window lookup matched name only: switching by the API id matched nothing and silently kept the previous model's window.

  • A revision that moves a value the client displays is broadcast (config_applied, issue #1374). A connected client never re-asks — the only frames carrying the effective vision are the pong and model_set, and a reload is neither (the TUI's pings are event-driven: startup, reconnect, rewind) — so before this, vision = true edited here left every client showing the previous answer until it happened to reconnect. Both directions move, and the false one is the dangerous half: a client that keeps showing img while the daemon refuses images is exactly the failure the segment exists to prevent. The frame carries the fields its reader uses (model, vision) plus the keys that moved, and no others: context_window was in the set while nothing displayed it (issue #1384) — the frame's reader reads type/model/vision/applied and the segment has no window field — so a window-only edit is now evidenced by the reload log line alone, the collector every daemon-local reloadable field has. A client surface that starts showing another field adds it here and to BROADCAST_ON_RELOAD; the tests beside the frame pin that membership in both directions. It deliberately carries no vision_source, because that key names how a /model switch resolved the value and a reload resolves nothing — the daemon already holds the resolved value. It is its own frame type rather than a reused model_set so that a broadcast the client did not ask for can never answer a /model request a GUI is waiting on.

  • base_url, api_key, max_tokens, temperature, max_tool_rounds, context_window, auto_compact_threshold, models, vision, stream_options, context_refresh_interval_ms are assigned for the next request; a stream already in flight is never rewritten.

  • A half-written or wrongly-typed file is rejected whole: the previous good configuration stays in force, one warning is logged, and the file is re-read on your next save.

A config.toml edit never restarts the daemon. The client used to compare this file's mtime against the running server's start time and SIGTERM→SIGKILL it when the file looked newer — which killed the running scheduler handlers (a live evolution cycle among them) and dropped every connected client, to apply an edit the daemon now applies itself. That branch is gone; a source change is the only thing that still restarts the daemon. Both sections are live: [llm] as described above, and [update] (enabled, delay_minutes) — the daemon's upgrade manager holds the same object the reloader writes to, so a change lands on its next 5-minute check. The upgrade interval stays hard-coded and is not a field of that section.

Verify from the daemon log (~/.emrg/logs/emrgd.log): every accepted edit logs one line naming the keys that moved —

config.toml reloaded: changed=max_tokens,temperature
config.toml reloaded: [update] changed=enabled
config.toml reloaded: model→gpt-4o (via the /model path)
config.toml change rejected (previous config kept): TOMLDecodeError: ...
config.toml change rejected (previous config kept): [update] delay_minutes is str, expected int

Update checking ([update] section): enabled = true|false (default true) enables the periodic GitHub release check, and delay_minutes (default 1440) is how long after a release is published it becomes eligible (set 1 for immediate). Both keys are hot-reloaded like [llm]: edit the file and the reloader assigns the new values within a couple of seconds, so the next upgrade check reads them — no daemon restart. That check runs every 5 minutes, and the interval is not configurable. The program only triggers: it never downloads an installer package, it starts an agent session (emrg-upgrade) that installs the equivalent of the release from the local evolution repo.

Terminal theme ([client] theme): the TUI used to name terminal-palette colours (cyan, green, magenta, red, dim) directly, so the terminal decided what they were and the accents were only readable on a dark background (rant 2026-10-14T14:48:32, issue #2246). theme is auto (the default), dark or light; anything else is read as auto — a colour is not worth refusing to start a client over. On auto the client asks the terminal which background it has: COLORFGBG when it is set (free, and only a palette index), else an OSC 11 query with a bounded timeout so a terminal that never answers cannot hang the client — and when neither says, today's dark appearance is kept. On a light background the accents are replaced by a contrast-checked palette and yellow/white — which cannot reach the ratio without changing meaning — are dropped to the terminal's own foreground rather than reinterpreted. The key is read by the client, not the daemon, so it is not hot-reloaded: a change takes effect when the next client starts.

Which shell tool runs ([sandbox] section): the dialect is decided by the platform, and the daemon mounts that dialect's executor. On Windows the daemon registers a pwsh tool; everywhere else it registers bash. Exactly one shell tool is ever registered, and the platform's is the only one that exists — Windows has no bash tool row at all (Git for Windows ships bash.exe under …\Git\bin while the installer puts …\Git\cmd on PATH, so "bash is on PATH on Windows" is not a fact to rely on). The system prompt is rendered from the tool that was actually registered, so it never advertises a dialect the platform does not run.

The executor's confinement is the OS process boundary (macOS Seatbelt, Linux bubblewrap, Windows ACL-restricted tokens): a write outside the session's working directory is refused by the kernel whatever language or subprocess attempts it, instead of by a parser that could not read a python3 -c. There is no switch: bash_tool_v2 and EMRG_BASH_TOOL_V2 selected the long-standing tool, whose confinement was a static scan of the command text, and P7 deleted that tool — a config.toml that still carries the key keeps working, and the key is ignored.

pwsh_path (default empty) names the PowerShell executable the Windows roster uses. Empty means the resolution chain decides, in the blueprint's order: the ProgramFiles PowerShell 7 install directory, then each PATH entry (a Microsoft Store install is an app-execution alias on PATH), then Windows PowerShell 5.1 at %SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe — which every Windows host has, so "PowerShell is missing" is not a reachable state there. A value you set is used verbatim.

Unlike [llm] and [update], pwsh_path is read once at startup and is not hot-reloaded: it decides which executable the registry's tool is built with, and the registry is constructed once. Change one and restart the daemon. Every shell tool receives the session's working directory as its authorization root, injected by the daemon rather than chosen by the model (workdir/workspace in a tool call cannot widen it).

[sandbox]
pwsh_path = ""              # Windows only: explicit pwsh.exe; empty = resolve (7 → PATH → 5.1)

The new path's boundary exists on macOS (Seatbelt), Linux (bubblewrap) and Windows (ACL-restricted tokens), and each platform reports how much it actually covers. On macOS and Linux confinement happens at the process boundary — the workspace root plus a private temp area — and the enforcement is reported as full; Linux needs bwrap installed and a kernel that allows user namespaces (a container needs --security-opt seccomp=unconfined --security-opt apparmor=unconfined --cap-add SYS_ADMIN), and without it the command is refused, never run unconfined. Windows reports partial, and that is its own honest claim, not a placeholder: Everyone must sit in both restricting lists for the restricted child to start at all, so an object granting Everyone write access stays writable, and an NTFS hard link can alias a granted file to a path outside the workspace. A granted directory must also be one the caller's own token can reach: under WRITE_RESTRICTED a child is checked against the token's own SIDs before the restricting list, so a workspace owned by someone else — Administrators, say — is refused before the capability ACE is ever consulted. Deployer-provided working directories are the running user's, which is the precondition; a workspace handed over by another account is not confinable this way, and the boundary test setup grants the caller's own SID for exactly that reason. Landlock remains the blueprint's second Linux rung and is not built here yet (design §1.5 B1).

A confined run cannot write your home directory, which is where uv, pip and npm keep their caches by default: measured at workspace-write, uv run fails (Failed to initialize cache at ~/.cache/uv), npm's ~/.npm cannot be created, and pip quietly disables its cache. So the tool points those four variables (XDG_CACHE_HOME, UV_CACHE_DIR, PIP_CACHE_DIR, npm_config_cache) at <temp>/emrg-confined-cache for the confined child — a directory inside a root the policy already grants, so nothing is widened. A variable you set yourself is never overridden, which is how you keep a warm cache: point it at a directory inside the session's workspace.

A task's frequency by time of day (schedule in ~/.emrg/tasks.yml): a record's single interval answers at every instant unless it states schedule, an optional list of windows:

- name: feishu-inbox
  interval: 10800        # the frequency outside every window
  schedule:              # optional; without it the task behaves exactly as before
    - days: mon-fri      # optional = every day; mon, mon-fri, mon,fri, fri-mon
      from: "09:00"      # quoted: unquoted, YAML reads 09:00 as a number
      to: "18:00"        # 24:00 allowed, and later than `from`
      interval: 900      # the frequency inside this window

Inside a window that window's interval answers; outside every window and between windows the top-level one does, so one instant never has two answers. Times are local time only — read with datetime.now(), with no timezone field and no DST rule — so the windows follow the host's clock; there are no public holidays and no make-up workdays, only the weekdays a window names (fri-mon wraps around the week). A wait that would run past the next edge is cut to it: a 3-hour weekend cadence that begins at 06:59 wakes at 09:00 rather than sleeping through the window it was about to enter, and a restart inside a window does not drop the task to the out-of-window frequency. Two windows of one day may not overlap, a window crossing midnight is written as two windows, and every field is validated when the file is read — a bad window is refused with the value it refused and the shape it expects. The daemon's create/update path and the task listing carry the same shape (effective_interval and next_window_boundary_in_seconds in list_tasks); until a client can edit windows, hand-editing this file is how to set one, and the scheduler picks the change up at the task's next wake.

🏗️ Architecture

┌─────────────┐    WebSocket (ws://)    ┌──────────────┐
│   emrg TUI  │ ◄─────────────────────► │   emrgd      │
│  (client)   │  TCP loopback + auth    │  (daemon)    │
│             │  token (emrgd.token)    │              │
│  • Chat     │                         │  • LLM loop  │
│  • Markdown │                         │  • Tools     │
│  • ToolCards│                         │  • Evolution │
│  • Autocomplete                       │  • Sessions  │
└─────────────┘                         └──────────────┘
  • emrgd — The daemon: runs the LLM tool-calling loop, manages sessions, drives evolution (a persistent background thread keeps thinking/evolving even while idle)
  • emrg — Your terminal: streaming markdown, command autocomplete, session browser
  • Skills — Dynamically loaded modules (browser harness, installers, etc.)
  • Memory — YAML frontmatter + Markdown files, auto-indexed, searchable

Project structure

emrg/
├── emrg/                   # Core package
│   ├── server/             # Daemon — LLM loop, tool execution, evolution
│   ├── client/             # TUI — python-tui based interactive chat
│   ├── gui/                # Electron GUI (non-developer entry point, Phase 3)
│   ├── tools/              # bash, read, write, edit, glob, grep
│   ├── skills/             # Dynamically loadable modules
│   └── __main__.py         # CLI entry point
├── tests/
├── .github/workflows/      # CI pipeline (pytest + conflict marker check)
├── MANIFESTO.md            # Design constitution
└── pyproject.toml

🧪 Development Workflow

git clone https://github.com/argszero/emrg.git
cd emrg
uv sync              # install deps
uv run pytest tests/ -v   # run tests (the count is measured live, not stored — see scripts/check-doc-count.py)
uv run python -m emrg     # launch TUI
# CI includes actionlint workflow gate (#444): workflow parse errors fail PR CI

Quick sanity checks:

uv run python -c "from emrg.client.app import run_client"   # import check
uv run python -m emrg --help
uv run --no-sync python3 scripts/check-rant-citations.py    # every citation names its public record

scripts/check-rant-citations.py answers one question about the instruction prose (the built-in task templates, prompts/*.j2, docs/gui-redesign.md): does every citation of a host rant name the public record that landed it? A rant timestamp indexes ~/.emrg/rants.jsonl on the machine that wrote it, so it is unresolvable for every other reader; a PR number stays resolvable. It exits 0 when the rule holds, 1 on a violation, and 2 when a file in the class cannot be read — 2 is not a pass. The class includes evolution_prompt.md; the red line around that file covers the running copy — the one resolved from the loaded module, i.e. <dir of scheduler.__file__>/evolution_prompt.md — so the repository copy's citations are swept like every other file's and the frozen-debt list is empty (the mechanisms that hold a site out of the rule are exercised by synthetic tests rather than by a real entry). --measure prints the whole inventory.

Electron GUI

cd emrg/gui
npm ci               # install deps (production: --omit=dev)
npm start            # launch GUI (auto-starts daemon)
npm test             # run Node tests (the counts are measured live, not stored — see scripts/check-node-test-count.py; integration runs in CI, local: npm run test:integration)

Packaging (installer builds)

Generated icon products (icon.png/icon-512/icon-256/icon.icns/icon.ico) are not committed — the repo keeps only the SVG design source (packaging/assets/icon.svg); CI generates them at build time (#688). When building installers locally (packaging/make-installer.sh / build-runtime.sh), run the generator first:

bash packaging/gen-assets.sh   # icon.svg → png/icns/ico (idempotent)

Renderer priority: rsvg-convert → Chrome/Chromium headless → sips (last resort, glow may be lost). Requires macOS iconutil for .icns (skipped with a notice on Linux/Windows).

The host's own render is the one that decides what the host sees. CI installs librsvg on every leg, so gen-assets.sh takes the rsvg-convert route there; a host without it takes the Chrome-headless fallback, and the two renders can differ (measured on 0.3.10, same version, two artifacts: the published pkg's icon.icns was alpha 0 at all four corners while the host's installed app was opaque white — the ring). Read the icon that was really built, from the artifact or from the app, instead of reading the source or trusting a green CI:

uv run --no-sync python3 scripts/check-icon-transparency.py <path>
# a .png, a .icns, or a directory: an .app bundle, an expanded .pkg, a dist/ tree

Exit 0 = every representation read is fully transparent at all four corner blocks; 1 = one is not, and the corner values are printed (that reading is the white ring); 2 = nothing could be read — no icon under the subject, or a container this tool does not open — never a pass. An artifact's icon is reached with pkgutil --expand <pkg> <dir>, then the component's Payload through gunzip -dc Payload | cpio -i -d; the Linux tar.gz carries no icon at all, so point the tool at an .app or an .icns rather than at that archive.

The release asserts this on the pkg it just built (issue #2283). The macOS leg of build-release.yml expands dist/artifacts/EMRG-*-macos-*.pkg, runs the command above on the payload it extracted, and fails the job on exit 1 and on exit 2 — while the pkg is still unsigned and unpublished, so the white ring cannot reach a release at all. A generated .icns being non-empty (test -s, the step that renders it) is not this question: an opaque ring is non-empty.

That step runs nowhere until a tag, so it was read from the workflow's own parsed YAML and run verbatim on a macOS host against the published v0.3.12 .pkg (2026-10-15): pkgutil --expand → cpio -i -d --quiet over the component Payload → the tool on the payload it extracted, rc 0 with all eight representations clear. The one option it uses that this repository uses nowhere else is settled by that same run: /usr/bin/cpio is bsdcpio 3.5.3 and accepts --quiet (the extraction returns 0).

To reproduce CI's reading at home, run the same route on the published artifact — same tool, same answer. Measured on v0.3.12 (EMRG-0.3.12-macos-arm64.pkg, sha256 fdc0c380e2b1f273e0c41391568dbf6409c81b232a5655343c955097a0323de3): every PNG representation of the shipped app reads alpha 0 at all four corners, rc 0.

gh release download v<ver> -R argszero/emrg --pattern 'EMRG-<ver>-macos-arm64.pkg' --dir /tmp/emrg-pkg
rm -rf /tmp/emrg-expanded
pkgutil --expand /tmp/emrg-pkg/EMRG-<ver>-macos-arm64.pkg /tmp/emrg-expanded
(cd /tmp/emrg-expanded/*.pkg && gunzip -dc Payload | cpio -i -d './emrg-gui/EMRG.app')
uv run --no-sync python3 scripts/check-icon-transparency.py /tmp/emrg-expanded/*.pkg/emrg-gui/EMRG.app

The rm -rf is not decoration: pkgutil --expand creates its destination and refuses one that already exists — measured on v0.3.12, a second run prints Error encountered while creating /tmp/emrg-expanded. Error 17: File exists and exits 1, a code this page otherwise reads as "the icon is opaque". The CI step is immune because it expands into a fresh mktemp -d.

The payload lists ._icon.icns (9798 B) beside the icon (612824 B), but the command above never produces it on disk: it is one of 64 AppleDouble entries in that payload and BSD cpio consumes them as extended attributes (the extracted icon.icns carries com.apple.cs.*), so the .app this recipe hands the tool holds the icon alone. Reading a directory that does hold a *.icns which is not an icns container names that file and measures the icon anyway; a tree whose whole .icns population is such files answers 2, never 0.

Does the version inside the artifact name the version the release claims? (#2286 — the host-side counterpart of the release job's Assert the artifacts declare the version the tag names step.) The chain asserted the version three times and never inside a product: tests/test_version_sync.py and bump-version.py --check compare the eight source declarations, check-release-tag.py compares the tag with the tagged tree, and check-release-published.py reads the release's metadata — draft, prerelease, asset names — without opening one. The artifacts' version.txt comes from the built tree (packaging/build-runtime.sh), and UpgradeManager.tick compares that installed file with target.lstrip("v"), so a release whose name and product disagree re-triggers the upgrade session on every tick, forever. Read the product itself:

uv run --no-sync python3 scripts/check-release-artifact-version.py 0.3.13 <subject> [<subject> …]
uv run --no-sync python3 scripts/check-release-artifact-version.py --from-tag v0.3.13 dist/artifacts/EMRG-*-linux-*.tar.gz
# a subject is a runtime .tar.gz, an .app bundle, an expanded payload directory,
# an .asar, or a version.txt

Exit 0 = every declaration read names the claimed version; 1 = one names a different one (both values printed — this is the case where the bundle was rebuilt and the packaged GUI dist was not); 2 = nothing could be read — a container this tool does not open, a path that is not there, a header that will not parse — never a pass. Two readings the hand-run got wrong, and the tool now gets right: the app's manifest is selected by its own package name, because a real app.asar holds eight package.json files and a dependency's version is not the product's; and a declaration that was read but claims nothing (the payload's 卸载 EMRG.app declares no version at all) is a note, not rc 2, while an unreadable one is never washed out by a readable sibling. The .pkg is reached the same way as the icon above — pkgutil --expand, then gunzip -dc Payload | cpio -i -d — and pointing this tool at the resulting directory runs the identical reading CI runs.

Does the product ship the prompt the release's tree carries? (The host-side counterpart of the release job's Assert the artifacts carry this tree's agent prompt step.) The version assertion above reads what a product declares; this one reads what it carries. Without it the same defect sits one layer up: a product built from an older tree passes every version check and hands the host the previous system prompt while the release names the new one. Read the shipped copy out of the product itself:

uv run --no-sync python3 scripts/check-release-prompt.py dist/artifacts/EMRG-*-linux-*.tar.gz
uv run --no-sync python3 scripts/check-release-prompt.py --file emrg/server/prompts/system.j2 <expanded-payload>
# a subject is a runtime .tar.gz / .tgz, or an expanded payload directory

Exit 0 = every product carries this tree's copy byte for byte; 1 = one carries a different text — both character counts are printed, with the first differing line of each side; 2 = nothing could be read (a container this tool does not open, and it then names the command that reaches one; an archive with no such member; a path that is not there) — never a pass. The prompt does not live in app.asar: that bundle is the GUI (emrg/gui/package.json's build.files), while the server is copied into dist/runtime by packaging/build-runtime.sh and travels at runtime/source/emrg/server/prompts/system.j2. Measured on the published EMRG-0.3.21-linux-x86_64.tar.gz: one such member, 24,558 characters — the text from before #2397. A .pkg or an AppImage is reached the same way as the two readings above (pkgutil --expand, --appimage-extract), and pointing this tool at the result runs the identical reading CI runs.

The host-side counterpart of the installer's compiled-extension guard (#1544). The macOS build signs the app with com.apple.security.cs.disable-library-validation so a pip-installed C extension can load; the installer asserts that entitlement's shape, and this command asserts its effect — a wheel's .so really imports under the bundled interpreter. Run it before reporting "packaging is broken", or after an install, to tell the two apart:

python3 scripts/check-extension-load.py                 # default: ~/.emrg/install/bin/python
python3 scripts/check-extension-load.py --python <path> --keep

Exit 0 = a wheel's compiled module loaded; 1 = it did not (the load decides, not the codesign reading — an interpreter without the entitlement can still load, and vice versa); 2 = could not measure (no such interpreter, it cannot run, or no wheel for this platform) — never a pass. CI runs the same check as step 15 of packaging/smoke-test.sh, so host and CI ask one question in one place.

The host-side counterpart of the release's notarize step. The macOS job of build-release.yml is the only place notarization is verified, and the tag is its only trigger — so a credential Apple refuses fails that step in seconds, skips the release job because of it, and publishes nothing at all (measured on v0.3.8, run 36956685533; the other three platforms built green). Ask at home first, with the same three variables the workflow fills from secrets:

APPLE_ID=<id> MACOS_NOTARY_APP_PASSWORD=<app-specific-password> \
    MACOS_NOTARY_TEAM_ID=<team> \
    uv run --no-sync python3 scripts/check-notary-credentials.py
# keep them out of the shell history instead:
#   uv run --no-sync python3 scripts/check-notary-credentials.py --env-file ~/.emrg/notary.env

Exit 0 = Apple accepted the credentials; 1 = Apple refused them, and Apple's own reply is printed (usual causes: an expired or revoked app-specific password, an Apple ID or team ID that does not match, or a Developer Program agreement waiting to be accepted); 2 = the exchange did not complete — or the --env-file named could not be read, a typo or a file not created yet; the message says which, and in both cases no verdict was reached — never a pass, and CI must fail on it too. After fixing, resume the run that failed with gh run rerun <run-id> --failed: a tag already pushed is not re-pushed.

The step's two failure modes are told apart by duration, not by the exit code: a refused submission dies in seconds, while a notarization verdict takes minutes, exits 0 and reports status=Invalid (the step parses that status and fetches Apple's rejection log for it). Notarize pkg names the preflight in its own ::error:: when the submission is refused, so the remedy arrives with the failure.

The host-side counterpart of the workflow gate. CI runs actionlint over .github/workflows/ on every push (in test.yml: the pinned release binary, downloaded into $RUNNER_TEMP with the version taken from that step's ACTIONLINT_VERSION — before it, the rhysd/actionlint@v1.7.12 Docker action, which built its own image at run time and reddened the whole test leg whenever Docker Hub throttled anonymous pulls, issue #2019. The gate exists because #441 reached a push with a workflow that referenced the secrets context in an if:), and a workflow edit used to have no local way to be checked first: the instructed actionlint .github/workflows/*.yml is a command this host does not have (measured 2026-10-04: actionlint, shellcheck, node and npm are all absent from PATH here, while brew and docker are present). So the check is one command, and it answers honestly when the tool is missing rather than looking like a pass:

uv run --no-sync python3 scripts/check-workflows.py          # every workflow in this checkout
uv run --no-sync python3 scripts/check-workflows.py --root <checkout>

It reads the pinned version out of the workflow file itself (no second copy of the number) and prints both that and the build it found. Exit 0 = the pinned actionlint ran over every workflow file and reported nothing; 1 = it reported a problem, quoted verbatim; 2 = could not measure — never a pass, with the reason: no actionlint on PATH (the remedy, including the pinned version, is printed), no workflow file in the tree, a tree whose workflows run no actionlint, or a local build that is not the pinned one — a different build's clean answer is not that gate's verdict, while a finding from any build is still exit 1. Install with brew install actionlint (check the version it gives you; the releases page has the pinned build). CI remains authoritative: this runs the same tool at the same version over the same files, not a second implementation of its rules.

The host-side counterpart of the release's retry bounds. The release workflow retries several network steps a bounded number of times, and the number an operator reads in the log is the one they triage a lost release on. Two of those loops used to carry a second copy of their own bound — the loop counted with one number and the sentence printed another — and nothing in the suite read a message, so a widened loop left the sentence stale after the existing guard had been satisfied (issue #2210). One command answers whether any loop still does:

uv run --no-sync python3 scripts/check-retry-bounds.py          # every workflow in this checkout
uv run --no-sync python3 scripts/check-retry-bounds.py --root <checkout>

It measures the derivation, not the spelling: for every retry loop, each value the loop presents as its bound — after the counter's slash, before the word "attempt(s)", or as the right operand of the counter's own -lt/-le — must equal the bound the loop iterates, with $VAR resolved from the nearest assignment above it. A stale literal is caught, and so is a variable that disagrees. Exit 0 = every claim agrees; 1 = one disagrees, and the file, the line, the claim, the bound and the sentence itself are printed so the fix is right there; 2 = could not measure — never a pass, with the reason: no workflow file, no retry loop at all, a bound variable with no assignment above it to read, or a loop that names the counter in a spelling the reader does not classify — for attempt in 1 2 3 4 with do on the next line, or an until loop, or for attempt in $(seq 1 4). That last reason is the point of the exit-2 contract rather than a corner of it: a loop whose bound cannot be read is exactly where the guard must not answer OK, and it prints the header and its line so the reader can rewrite the loop or teach the reader the spelling.

The readable path to a failed run's cause. gh run view <id> --log and --log-failed answer 0 bytes with exit 0 on a current host for every run, green or red (measured 2026-10-04 on gh 2.58.0: v0.3.8's failed build 36956685533, v0.3.7's green build 36658495939, and a recent Test run all return nothing on stdout and stderr, rc 0) — a silent empty answer that reads exactly like "this run has no log". gh api is unaffected (the same job's log is 689 KB), so:

uv run --no-sync python3 scripts/read-run-failure.py <run-id>            # the failed job, its step, its output
uv run --no-sync python3 scripts/read-run-failure.py <run-id> --tail 60  # more of the failing step's block

It prints the job the payload marks failed, the step that failed (only the payload names it — a step that dies silently leaves no line in the log at all, which is exactly the notarize case above), the runner's ##[error] annotation, and the failing step's block up to that annotation rather than the log's tail — the tail is the runner's own teardown (17 lines of Post job cleanup on the run above, which bury the cause). Exit 0 = a cause was printed; 1 = the run has no failed job; 2 = the question could not be answered (bad id, gh failed, no jobs listed, or a failed job's log came back empty) — never a pass.

"Failing" means a job concluded failure, not the run. A workflow run's own conclusion is an aggregate over its jobs, and GitHub counts a cancelled job as a failed run — so a run that never got a runner (a job that sits 15 minutes with 0 steps and is cancelled) reports failure while no job of it ever judged the tree. check-merge-freshness.py therefore asks the run's jobs when a run is non-success, and reports the state it finds: a job concluding failure is the red verdict (and the cause-reading documented above can name its cause), while a run whose non-success jobs are all cancellations is reported as a missing verdict whose remedy is a re-trigger — the same action a head with no run gets. Measured 2026-10-06: without that split, the row said "a failing verdict, not a stale one; re-running will not make it fresh" about a cancelled run, and the reading its own remedy handed over answered "no failed job … nothing to explain" (exit 1) — a verdict-shaped sentence about a run that reached no verdict, with a remedy that could not produce a cause. When the jobs list itself cannot be read, the coarse reading stands and the report says that half is unmeasured, never which of the two it would have been.

It works without a GitHub token too, which matters because gh refuses every call when it is unauthenticated ("please run: gh auth login") while the same paths answer anonymously: the tool falls back to api.github.com with no credentials, prints which channel answered, and reads the runner's annotations from the job's check run — so a tokenless host still gets the failed job and the failed step. The job log is the one part GitHub will not serve anonymously (403), and the report says so per job instead of reporting no cause.

A crash is a measurement error, not a verdict. A tool that dies on an unhandled exception exits 1, and 1 is a verdict in these tools' own tables — check-merge-freshness.py reads it as STALE (whose documented remedy is a re-merge and a push that voids every standing vote), and review-queue.py does not define 1 at all. Measured 2026-10-06 with a sibling left unparsable: those two exited 1 with no verdict-shaped line, so a caller reading the code read a verdict that was never reached. So a crash has to leave as the tool's own "could not measure" answer, and the family pinned by tests/test_a_crash_is_a_measurement_error.py — the gates that load a sibling by file path, and the ones that answer by running another tool as a subprocess — does it in one place: each ends its __main__ block in _entry(), which runs main() and, if it raises, prints the traceback with a <tool>: could not measure - <Type>: <message> line and answers 2. A gate that loads no sibling reaches the same rule by its own route and says so where it does — check-merge-tree-health.py:_merged_tree_sha raises MeasurementError "because an unhandled exception leaves this tool as exit 1 - the code that means 'a clean merge landed an unhealthy tree'". The load sites no longer carry a if spec is None or spec.loader is None guard either: importlib.util.spec_from_file_location returns a spec and a loader for a path that does not exist, so that branch could never fire — tests/test_a_crash_is_a_measurement_error.py pins both halves, pins the entry point identical across the family (one rule, not a copy per tool free to drift apart), and derives the family from the source (a spec_from_file_location call in code), so a gate that loads a sibling and is not in that list fails the module instead of sitting silently outside it.

CI runs tests and checks for conflict markers automatically via GitHub Actions (.github/workflows/test.yml).

Self-evolution from source: the evolution workspace expects the repo at ~/.emrg/evolution/emrg. Packaged installs self-heal (clone on demand + auto-bootstrap projects/tasks); source installs should clone there explicitly if you want the evolution daemon to work on this repo.

A test that derives its expectation from what it wrote is a POSIX assumption

test-windows runs the same suite on a shell where text-mode writes translate every line ending, so an expected value taken from the Python string that was written is wrong there while staying green locally. This has cost two CI reds in this repo, both times on a brand-new test that passed on macOS:

  • assert log.stat().st_size == len("previous run\n") — the file holds one more byte per line on Windows, so the assertion is about the platform, not about the code. Read the expectation back from the artifact instead (mark == log.stat().st_size), or assert a property rather than a literal: the mark must be a point in the file, so the size grows by exactly the appended line's length read through the same reader.
  • A not in check against multi-line source text is satisfied by the line endings alone on a CRLF checkout, so the guard silently stops guarding while staying green — normalise (text.replace("\r\n", "\n")) before any substring assertion that spans lines. A silently passing guard is worse than a red one.

Neither needs a Windows machine to reproduce: write a file containing "line\r\n" and drive the code path under test with it. Local green is a statement about the platform you are standing on; when a test's subject is bytes, sizes or whole-file text, ask what the other platform's write path does to it.

A test file the guards cannot see yet — green locally, red in CI

Several guards here decide what to read by asking git for the tracked files (git ls-files): the encoding rule (test_script_decode_is_locale_independent.py), the CRLF rule, the conflict-marker rule, the doc-count rule, scripts/check-doc-count.py, and others of the same shape. That is deliberate — an index cannot go stale — and it has one consequence: a new file that is not in the index is read by none of those rules.

Measured 2026-09-18 (cycle cyc20260918-105223, PR #1368): a new test file was written, the full suite ran green in its worktree, the branch was pushed — and CI failed four minutes later on the encoding rule, naming a subprocess.run(..., text=True) call in that file. The local run had answered about a tree the file was not part of.

The remedy is one command: git add a new file before trusting a local suite run. Staging is enough — git ls-files lists staged files. tests/test_the_index_derived_scans_reach_new_files.py now fails while a first-party .py file under emrg/, packaging/, scripts/ or tests/ is untracked, so the local run cannot report a verdict over source those rules cannot read; a file staged and then edited (the AM state) is a different question and no guard answers it.

❓ Extended FAQ

What LLMs work with it?
Any OpenAI-compatible API. Tested with DeepSeek and OpenAI. Works with Anthropic (via proxy), Ollama, vLLM, and other local models.

Why does the Windows installer show "Unknown publisher"?
The Windows installer is not Authenticode-signed (that certificate costs money to obtain and is not procured yet), so SmartScreen shows "Publisher: Unknown" and may block the run. This is a standard Microsoft security prompt for newly released/unsigned software — it does not mean the file is bad: EMRG is fully open source (MIT) and auditable. To proceed: click "Keep" on the browser prompt; click "More info → Run anyway" on the run prompt; or right-click the exe → Properties → check "Unblock". The macOS installer is signed + notarized (v0.2.7+) and has no such prompt.

Can it break itself?
Every change is validated by pytest and an import check before commit. Failed changes are discarded. The worst case is a rollback.

How is this different from Claude Code or Codex?
They're products. EMRG is an experiment in closing the loop — the AI improves the AI. Also: fully open source, no vendor lock-in, and you control your data.

🐛 Troubleshooting

Why is my evolution cycle running read-only?

Every cycle gets one bash tier: read-only, workspace-write (the default), or danger-full-access. The tier is decided per cycle and is keyed on whether discarding the working tree would lose anything, not on whether the tree is dirty (issue #1237). When the task's source repository has uncommitted changes:

  • nothing exists only there — every change is already published at that path in a ref git holds (HEAD, the upstream tip, or any other ref tip: a branch you already pushed, a tag, a refs/cdrain/prNNNN tip) — so the daemon converges the tree itself and the cycle keeps its configured tier;
  • something exists only there — an untracked file, a staged addition, a conflict, a modification outrunning every ref tip, or a commit no other ref reaches — so the cycle is forced down to read-only whatever the configuration says (community issue #979), and the paths that caused it are named in the log.

Why the distinction matters more than it looks: read-only refuses the very git verbs that could clean the tree, so the state that triggered the downgrade also disabled its only exit. Measured cost of one instance of that loop: 33 consecutive zero-commit cycles, whose verified output had to be parked outside the repository. And the dirt in it was not work at all — of what git reported, the modified files were byte-identical to upstream's own blobs, i.e. the second bullet above had been applied where the first was true.

Recovering a tree

A cycle recovers itself. At the start of every cycle the daemon asks the question above, and when the answer is "nothing exists only here" it converges the tree itself, before the cycle's first tool call — reversibly (stash, HEAD unmoved, receipt in the git state dir). The exit from a downgrade is therefore the daemon's own action, not a command a human has to be present to run: a guard whose only exit is manual strands the tree, which is the shape of the measured 33-cycle loss above. Nothing is asked of the host.

Read-only blocks the destructive shapes the guard recognises: redirects to anything but /dev/null, rm / rmdir, mv / cp destinations, truncate / tee / shred, sed -i, find -delete, git ... --output=<f>, and every git subcommand that is not on the read allowlist (add / commit / stash / checkout / switch included). The recognised shapes are blocked wherever the target lives, inside the workspace or outside it — only /dev/null is exempt. It is a static scan, not an OS boundary (enforcement="partial"), so a command outside that shape list (mkdir, touch, an interpreter writing a file) still runs and a downgraded cycle can leave scratch files behind. The decision is per command string, not per statement: one blocked shape anywhere in the call refuses the whole call. Reads are unaffected — including the read-only git verbs the daemon's own diagnosis uses — so the problem can still be diagnosed from inside a downgraded cycle; only the repair needs a writable tier.

Diagnose before you repair

"Modified" does not say which of four things disagrees, and only some disagreements are safe to discard. Compare the content ids:

cd ~/.emrg/evolution/emrg
git hash-object <path>            # working tree
git ls-files -s <path>            # index
git rev-parse HEAD:<path>         # HEAD
git fetch origin master
git rev-parse FETCH_HEAD:<path>   # master

Then the decisive question — is the branch simply behind?

git merge-base --is-ancestor HEAD FETCH_HEAD \
  && echo "HEAD is behind master: nothing unmerged is at risk"

That is the common case in a workspace whose change was merged upstream after the local checkout: the working tree holds master's content while the index still matches the older HEAD, and the "uncommitted work" is already published. The daemon's own recovery action clears the dirty state reversibly without moving HEAD; the sequence below is for when you also want to move the branch onto upstream. Recover with:

git fetch origin master
git checkout -f -B master FETCH_HEAD

A plain git checkout master will not do it — fetching does not move your local branch, so that checks out the commit you are already on, exits 0, and leaves the tree dirty.

-f is the right tool for tracked modifications only. Measured on the other two geometries: untracked-only dirt survives it (exit 0, still dirty, so the next cycle downgrades again) — use git stash -u, recoverable with git stash apply --index, not git clean -fd, which destroys the content; and an untracked file whose path the target commit also adds gets its content silently replaced. Adding the path to .gitignore does not help, because that edit is itself an uncommitted change.

Try it without -f first. Git refuses exactly where -f would discard something, and the refusal names the file — Your local changes to the following files would be overwritten by checkout: <path> for a tracked modification, and The following untracked working tree files would be overwritten by checkout: <path> for an untracked path the target commit adds. Add -f only once that refusal is gone, or once the content it names is somewhere else.

The override

An audited escape hatch exists: EMRG_TASK_DIRTY_OVERRIDE takes a comma-separated list of task names, or *, and the downgrade is skipped with a receipt in the daemon log.

The daemon inherits its environment from the process that starts it, so set the variable on the command that starts the daemon:

EMRG_TASK_DIRTY_OVERRIDE=emrg-task emrg server restart

Make it an explicit decision — read git status and git diff first. The guard exists because a cycle's bash tool can otherwise overwrite work you have not committed, and the variable disables that protection for the named tasks.

When a fix seems not to have taken effect

Two gaps get mistaken for a broken fix:

  • merge ≠ running — a guard fixed on master protects nobody until a release ships it.
  • install ≠ running — after the installer updates ~/.emrg/install/source, a daemon that is already running keeps executing the old code until it restarts.

Check emrg -v and when the daemon started before re-reading the source.

emrgd failed to start within N s

Whichever entry point you started spawns the daemon and waits for it to accept connections. When that wait runs out, the error carries what is known about the attempt: whether the child is still running, its exit code if it is not, the daemon log lines this attempt appended, and the contents of ~/.emrg/logs/emrgd-start.err (the child's own stderr, which is where a failure before logging starts can be read at all).

Both entry points read the same variable, EMRG_START_TIMEOUT, in seconds: the terminal client (emrg, emrg/client/daemon_manager.py) and the GUI (emrg/gui/daemon_client.js). Their defaults differ because the GUI's was already its own — 4.5 s (15 polls, 0.3 s apart) for the client, 5.0 s for the GUI — and neither changes; what the variable adds is the ability to raise either one. Set it on a client (for the terminal) or in the environment the GUI is launched from:

EMRG_START_TIMEOUT=30 emrg

A value is a plain decimal number of seconds — digits, an optional sign, an optional fraction and exponent (30, 4.5, .5, 1e2). Anything else is ignored with a warning in that entry point's own log and the default is used: a typo in a tuning variable must never be the reason a start fails. The refusal is deliberate about the shapes that merely parse: 0x10, 1_000 and full-width or Arabic-Indic digits are all refused, so the same value cannot mean 16 seconds to one entry point and 4.5 to the other (1_000 would be a sixteen-minute wait if it were accepted). Both entry points read one shared list of allowed values in their tests — tests/data/start_window_shapes.json — so a divergence in either resolver is a test failure rather than a surprise.

Two facts make the window cheap to raise: a child that has exited is reported on the first poll with its exit code, so the window only ever bounds a child that is alive but not yet listening; and the reported bound is one the loop really applied, with a floor of one poll on both sides. The two report it differently, and both are honest: the client quantises the window to whole polls and names that product (1.0 → 0.9s), while the GUI's loop is deadline-based, names the deadline it enforced and may therefore overshoot it by up to one poll (1.0 → 1.0s reported, ~1.2 s actually waited).

Why is a write blocked at workspace-write?

workspace-write is the default tier, and it allows a write only when the target resolves inside one of three roots:

  • the injected workspace root (the project the task runs in),
  • the OS temp root tempfile.gettempdir(),
  • the trusted data roots — ~/.emrg/evolution/.emrg/, the evolution module's own records and scratch, trusted because its records live outside the workspace it is given (issue #1093).

Every other absolute path is refused, and the daemon's own state files are refused inside any root (~/.emrg/config.toml is protected; a write to ~/.emrg itself is refused outright, because it can erase the daemon's data).

Measured 2026-09-16 against master: inside the workspace ALLOW, $TMPDIR/… ALLOW, /tmp/… BLOCK, /private/tmp/… BLOCK, /var/tmp/… BLOCK, /dev/shm/… BLOCK.

A relative target is resolved, not assumed

echo x > out.txt is allowed while the resolved target stays under the directory the command runs in — the workspace root, or the daemon's own cwd when the caller injected none. A target that climbs out with .. is refused, including the spelling that arrives through a variable the command itself assigns (issue #1353):

⛔ workspace-write sandbox: blocked write to relative target '../escaped.txt':
it resolves to '/…/escaped.txt', outside '/…/ws', the directory the command runs
in (issue #1353)

A climb that returns to the workspace (echo x > ../ws/out.txt) is still allowed: the test is on the resolved target, not on the spelling, so it cannot turn sub/../out.txt into a refusal.

A cd that stays inside the workspace moves the write site with it — the target is joined onto the directory the shell is in when it runs, not the one it started in — so cd sub && echo x > ../back.txt writes <ws>/back.txt and is allowed, while cd sub && echo x > ../../worse.txt climbs out of <ws>/sub and is refused (issue #1370). Proving which directory that is costs the shapes that cannot prove it, and those keep the refusal they had: a ; or || chain (the cd may have failed, and then the shell never moved), a ( … ) group or a pipeline (the cd may be in a shell of its own), a target written before the move, and — since issue #1357 — a destination only the shell could finish:

⛔ workspace-write sandbox: blocked write to relative target 'f': the command runs
it after changing directory to '$D', which is not a directory this workspace can
place it in (issue #1244)

A destination that still needs expanding is refused rather than joined onto the workspace, because the join would read it as inside: D=../outside && cd "$D" && cat > f, a loop variable over a directory, and cd "$(mktemp -d)" were all allowed while the shell writes outside the workspace. The class is "anything left to expand after the environment and the command's own assignments have been applied" — an unknown $NAME, a ${NAME:?}, a $(…) or its backquote spelling. The price is real and stated: a legitimate computed move (cd "$(git rev-parse --show-toplevel)", a directory a read filled in) is refused the same way, and so is cd "$(mktemp -d)" even though mktemp -d lands in the allowed temp root — where it lands is exactly what the text does not say. If you hit this, spell the write target absolutely, or cd to a path the daemon can read from the text.

$TMPDIR, not /tmp

The tier means tempfile.gettempdir() — $TMPDIR, which is /var/folders/<…>/T on macOS and /tmp on Linux. So /tmp is the temp root on Linux but not on macOS, and the refusal names the target without naming the rule:

⛔ [sandbox:workspace-write enforcement=partial] workspace-write sandbox:
blocked write outside workspace '/tmp/emrg-e2e-probe.txt' — command not executed

Use $TMPDIR (mktemp -d is the portable way) rather than a hardcoded /tmp in anything a sandboxed task runs. Nothing promises /tmp is writable, so the block is over-cautious rather than wrong — a boundary is not widened without the host's decision.

Two more properties are worth knowing before debugging a refusal, both measured the same day:

  • per call, not per statement — one blocked target refuses the whole call. echo a > inside.txt; echo b > /etc/x creates neither file: the guard decides before the shell starts, so the legal half never runs either.
  • the same /tmp target is refused by both tiers, for opposite reasons — read-only refuses the shape wherever the target lives (a redirect to anything but /dev/null), workspace-write refuses the target wherever it is written from. A refusal reason is how you tell which one you hit.

📜 License

MIT — see LICENSE for the full terms and MANIFESTO.md for the philosophy behind the code.