Skip to content

Commit fa1ec2a

Browse files
arena-ai-coding-agent[bot]hyperpolymatharena-agent
authored
Quarantine fleet mutations pending directive enforcement; audit repository policies (#571)
## Summary Safety-first containment and evidence, **not activation of the fleet**: - Quarantine main mutation paths until repository-local directives are enforced: dispatch, live fixes, findings processing, rule/issue publication, fake deployment readiness, and RRA CLI mutations. - Remove the Rust fleet CLI's simulated-success execution. - Add containment tests and run them in the E2E workflow. - Add read-only, commit-pinned directive census tooling with offline regression tests. - Document wiring defects and a census of 352 visible repositories: 746 bot-directive files and 607 AGENTIC candidates, including nested projects/templates; 512 unique content blobs fetched and hash-verified. - Preserve evidence metadata and a per-repository matrix. No raw directive bodies or credentials are included in the census artifact. ## Why The audit reproduced successful fix outcomes with no target checkout or executed fixer, and zero-coverage supervised scans returning success. Existing directives contain protected paths, draft-PR-only requirements, approval gates, data-preservation rules, legacy layouts, and retired exceptions. Repairing execution before enforcing these constraints risks destructive estate-wide changes. ## Operational impact / review carefully - This deliberately blocks the guarded entry points; shell quarantine exits with `78`, Rust guards return errors. Scheduled learning/processing jobs using this revision may fail visibly instead of claiming success. - Dispatch is blocked even with `--dry-run` because that path writes outcome/heartbeat state. - There is no environment-variable bypass. Removing the guards requires a reviewed change. - **This is not an estate-wide shutdown or a complete policy engine.** Direct fix scripts, library callers, enrollment writers, external apps, existing services and old binaries remain outside these CLI guards. - No target repository directives were rewritten and no repository was approved for mutation. No auto-merge is requested. - The proposed reusable policy resolver is follow-up work and is **not implemented in this PR**. ## Evidence and limitations Start with: - `docs/AUTOMATION-QUARANTINE.adoc` - `audits/fleet-wiring-2026-09-26.md` - `audits/directive-census-2026-09-26/REPORT.md` The census's `evidence.json` (~2.5 MB) accounts for most of the diff and contains paths, pinned revisions, hashes and retrieval metadata. Human semantic review was selective, not an approval of all fetched content. `julia-ecosystem` returned a truncated tree; private/inaccessible repositories, submodule contents, local-only guidance and deployed enforcement remain unverified. ## Validation - [x] `bash tests/automation-quarantine-test.sh` - [x] `bash tests/directive-census-test.sh` (offline fixtures; no GitHub calls) - [x] Shell syntax checks for changed/new scripts - [x] `git diff --cached --check` - [x] `bash tests/e2e.sh`: **36 passed**, after installing the existing PyYAML test dependency in a temporary directory - [ ] Rust compilation/tests: **not run locally; cargo/rustc unavailable**. Rust guard tests were added and require a Rust-enabled environment before merge. No executor, deployment or estate-wide workflow was enabled to perform validation. Co-authored-by: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
1 parent ce89763 commit fa1ec2a

19 files changed

Lines changed: 64768 additions & 56 deletions

File tree

‎.github/workflows/e2e.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ jobs:
2020
- uses: actions/checkout@v7.0.1
2121
- name: Install PyYAML (for workflow YAML validation)
2222
run: pip install pyyaml --quiet
23+
- name: Verify automation quarantine
24+
run: bash tests/automation-quarantine-test.sh
2325
- name: Run E2E tests
2426
run: bash tests/e2e.sh
2527
- name: Verify canonical descriptile policy repair

‎.machine_readable/bot_directives/coverage.a2ml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,3 +60,13 @@ coverage-percent = 0
6060
# easy-low-completed = 2
6161
# hard-low-deferred = 4
6262
# assessment = "Correctly prioritised — all MUST items addressed before COULDs"
63+
64+
[coverage.visited.automation-quarantine]
65+
date = "2026-09-26"
66+
ring = 1
67+
notes = "Audited main fleet execution paths and local directives. Added unconditional quarantine to main mutation entry points; no deployment or external repository changes. Shell containment tests and 36 existing E2E checks passed. Rust tests added but not run: cargo/rustc unavailable. This is not a complete estate policy audit."
68+
69+
[coverage.skipped-musts.directive-enforcement]
70+
priority = "P0"
71+
issue = "Inventory existing per-repo directives read-only; implement policy semantics and gate every mutator before lifting quarantine. Direct scripts, library APIs and deployed services remain outside these CLI interlocks."
72+
discovered = "2026-09-26"

‎.machine_readable/bot_directives/debt.a2ml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,10 @@ last-updated = "2026-03-24"
4848
# effort = "easy"
4949
# impact = "low"
5050
# discovered = "2026-03-23"
51+
52+
[[debt.should]]
53+
component = "automation-policy"
54+
issue = "Complete directive inventory, owner-reviewed precedence and per-operation authorization; qualify all direct entry points and deployed services. Quarantine must remain until P0 coverage item is resolved."
55+
effort = "hard"
56+
impact = "high"
57+
discovered = "2026-09-26"

‎.machine_readable/descriptiles/STATE.a2ml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,11 @@ actions = [
4545
[maintenance-status]
4646
last-run-utc = "2026-04-05T22:00:00ZT00:00:00Z"
4747
last-result = "unknown" # unknown | pass | warn | fail
48+
49+
[automation-quarantine-2026-09-26]
50+
status = "local-branch-only-not-deployed"
51+
reason = "Main paths do not reliably enforce repo-local directives; audit reproduced false fix success and zero-coverage scan success. Earlier operational claims above are historical, not current verification."
52+
changes = "Main shell mutation paths blocked with exit 78; Rust fleet simulation removed; RRA CLI mutations blocked pending policy qualification."
53+
validation = "Shell containment tests passed; existing E2E 36 passed after supplying PyYAML. Rust tests not run: toolchain unavailable."
54+
next = "Read-only estate directive inventory and explicit policy contract. Do not enable execution or increase permissions before local restrictions are enforced."
55+
evidence = "docs/AUTOMATION-QUARANTINE.adoc; audits/fleet-wiring-2026-09-26.md"
Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
2+
# Read-only repository directive census — 2026-09-26
3+
4+
## Decision
5+
6+
**Keep the automation quarantine. No repository is approved for mutation by this census.** Local restrictions are widespread and substantive. A single default policy or a check for a directory's existence would lose important protections.
7+
8+
This census used GitHub read operations only. It did not check out remote branches, run repository code, enroll repositories, modify directives, post issues, open PRs, dispatch workflows, grant permissions, or deploy anything. Only audit tooling and evidence were added to the local working branch.
9+
10+
## Coverage
11+
12+
| Measure | Observed |
13+
|---|---:|
14+
| Repositories returned by the authenticated account's owner listing | 352 |
15+
| Public / non-archived entries in that listing | 352 / 352 |
16+
| Complete recursive tree responses | 351 |
17+
| Truncated tree responses | 1: `julia-ecosystem` |
18+
| Candidate guidance files, including nested projects/templates | 5,702 |
19+
| Files under bot-directive directories | 746 |
20+
| `AGENTIC` candidate files | 607 |
21+
| Repositories with a bot-directive candidate anywhere | 153 |
22+
| Repositories with root `.machine_readable/bot_directives/` files | 147 |
23+
| Repositories with root legacy `.bot_directives/` files | 4 |
24+
| Repositories with both root layouts | 1 |
25+
| Repositories with neither of those root directory layouts observed | 202 |
26+
| Canonical root `.machine_readable/descriptiles/AGENTIC.a2ml` observed | 49 |
27+
| Unique content blobs retrieved and byte-hash verified | 512 |
28+
| Directive/AGENTIC file occurrences covered by those retrieved blobs | 1,353 / 1,353 observed |
29+
| All candidate file occurrences sharing retrieved blobs | 2,544 |
30+
31+
The remaining related-guidance files are path/hash inventoried but not content-fetched. Human semantic review was selective, focused on the orchestration repos and unusual restrictions below. **Retrieval is not semantic validation.** No count implies that nested/template guidance applies to the whole parent repository.
32+
33+
The inventory includes dot-prefixed repositories such as `.github`. It does not prove all private/inaccessible repositories were enumerated. Direct probes of `hyperpolymath/.gitbot-fleet` and `hyperpolymath/.git-private-farm` could not resolve those names with this session; absence and insufficient visibility cannot be distinguished. The name `.gitbot-fleet` can also denote local session storage rather than a GitHub repository.
34+
35+
Ten listed repositories contain submodule entries; this census records their commit IDs but does not recurse into them. Five candidate guidance entries are symlinks; they were not followed. The truncated `julia-ecosystem` tree and any unpublished/local-only instructions remain explicit coverage gaps.
36+
37+
## Evidence deliverables
38+
39+
- **`repositories.tsv`**: one row per repository, pinned commit, coverage status, directive-layout counts, and `not_evaluated` authorization.
40+
- **`evidence.json`**: candidate paths, Git blob hashes, modes, scope hints, retrieval status, SHA-256 for retrieved content, tree completeness and submodule entries. No credential values or raw directive bodies are included.
41+
- Raw tree/blob evidence remains outside the checkout at `/home/user/fleet-directive-census/`; it is not vendored into Git.
42+
43+
Remote snapshots are pinned individually, not a simultaneous estate-wide snapshot. The remote `gitbot-fleet` commit inspected was `ce89763a7f9b874288a909732bf21b3143c6afea`. Local working-branch quarantine changes are intentionally not represented as deployed remote policy.
44+
45+
## Findings that must shape enforcement
46+
47+
### 1. Real path exclusions and review-only requirements exist
48+
49+
[Axiology.jl fleet directive](https://github.com/hyperpolymath/Axiology.jl/blob/bdcc9ee8269be484c62336ff480b0ea0b19c464b/.machine_readable/bot_directives/gitbot-fleet.a2ml):
50+
51+
- `draft-PRs-only = true`.
52+
- `never-touch` includes ABI definitions, Zig FFI, `.machine_readable/`, workflows and proof specifications.
53+
- Scaffolding must not blindly follow stale template checks.
54+
55+
**Consequence:** a generic workflow fixer or metadata normalizer must be denied on protected paths, even when confidence is high. A generated migration must not rewrite its own authorization.
56+
57+
### 2. VeriSimDB has data-preservation and provenance rules
58+
59+
[Automaton directive](https://github.com/hyperpolymath/verisimdb/blob/f35ac59e8dab1877cb13a4c16de4eac6cf098d58/.machine_readable/bot_directives/robot-repo-automaton.a2ml) prohibits deleting database data/backups and requires persistent data and user datasets to be preserved absent explicit approval. It also defines cleanup cadence and an acknowledgement requirement.
60+
61+
[Cross-thread quarantine](https://github.com/hyperpolymath/verisimdb/blob/f35ac59e8dab1877cb13a4c16de4eac6cf098d58/.machine_readable/bot_directives/cross-thread-quarantine.a2ml) requires imported claims to remain `proposed-unverified` until checked against local specs. It expressly rejects certain language migrations and incorrect VCL architectural framing.
62+
63+
**Consequence:** neither estate-wide language policy nor generic cleanup recipes may silently override these instructions. This report is an audit proposal, not authorization to change VeriSimDB.
64+
65+
### 3. Hypatia's local instructions conflict with the fleet's old learning output
66+
67+
[Hypatia AGENTIC](https://github.com/hyperpolymath/hypatia/blob/9f2f62f5c9463c79b33a5ebf54372166ce56f349/.machine_readable/descriptiles/AGENTIC.a2ml) explicitly prohibits Logtalk, says rules are pure Elixir, and prohibits emitting JSON from its tooling. It requires evidence, reruns and behavioral tests, and declares automation rate/confidence constraints.
68+
69+
The audited fleet coordinator still generates Logtalk `.lgt` proposals and has a publication path to Hypatia. Its local quarantine now blocks that route; the census did not deploy that block.
70+
71+
The same AGENTIC file references retired `6a2` locations while acknowledging the newer `descriptiles` convention. **Do not automatically resolve this by deleting the local constraints or regenerating the file.** Separate the obsolete location reference from still-binding substantive prohibitions.
72+
73+
### 4. Cicd-squabbler explicitly defines conditional precedence, not a blanket override
74+
75+
[Gate-triage directive](https://github.com/hyperpolymath/cicd-squabbler/blob/c56bcc18ffb2bbda42aa149defc77adf6467b9a4/.machine_readable/bot_directives/gate_triage.a2ml):
76+
77+
- Automaton is primary **when present AND working**; squabbler is a standalone fallback.
78+
- Unavailable checks are triage cases, not self-awarded green results.
79+
- No autonomous gate deletion, dropping required contexts or transition to green.
80+
- Operator/channel applicability is distinct from a broken check.
81+
- The directive references `.machine_readable/bot_directives/rr_automaton/` override rules; that path is not present in the inspected fleet tree. Its intended location/contract needs owner confirmation, not invention.
82+
83+
**Consequence:** a policy-blocked job should be reported as blocked/unresolved, not successful. Nonzero quarantine exit is a refusal to execute, not proof that the target repository's code failed quality checks. Do not rewrite squabbler semantics to make the fleet appear green.
84+
85+
### 5. Echidnabot has a resolver, but errors can discard local mode selection
86+
87+
[ECHIDNA's echidnabot directive](https://github.com/hyperpolymath/echidna/blob/ba373a8b1d3309c561f7ff30e0197b21cc8bb185/.machine_readable/bot_directives/echidnabot.a2ml) requests `regulator` mode and a coverage threshold of 90.
88+
89+
The external [echidnabot resolver](https://github.com/hyperpolymath/echidnabot/blob/dbeee8ad278576b581b9729fb5bec84ea0dbb0e9/src/modes/directives.rs) fetches two canonical A2ML paths, parses `[bot].mode`, and falls back through database/daemon settings. Fetch errors are treated as no directive. A database mode equal to the built-in default cannot distinguish an explicit choice from an unset value. This module alone is not enforcement of all directive fields or a complete authorization engine.
90+
91+
The [echidnabot automaton directive](https://github.com/hyperpolymath/echidnabot/blob/dbeee8ad278576b581b9729fb5bec84ea0dbb0e9/.machine_readable/bot_directives/robot-repo-automaton.a2ml) permits low-risk edits but prohibits unapproved core changes and requires explicit rule approval. The fleet's SCM-only confidence reader does not cover this A2ML grant/deny contract.
92+
93+
Its [AGENTIC](https://github.com/hyperpolymath/echidnabot/blob/dbeee8ad278576b581b9729fb5bec84ea0dbb0e9/.machine_readable/descriptiles/AGENTIC.a2ml) retains a historical BoJ transport exception explicitly marked **superseded/inactive**. Reading the exception's old permissive text while ignoring its sunset would wrongly reactivate it. Deployment conformance was not verified.
94+
95+
### 6. Legacy layouts and nested scope must not disappear during migration
96+
97+
Root legacy directives remain in:
98+
99+
- `ipv6-site-enforcer`: SCM bot directives.
100+
- `network-dashboard`: SCM bot directives.
101+
- `odds-and-sods-package-manager`: legacy bot-specific A2ML plus canonical methodology files.
102+
- `proof-burrower`: legacy root methodology plus nested `affinescript` directives.
103+
104+
[Proof-burrower AGENTIC](https://github.com/hyperpolymath/proof-burrower/blob/dc210e9b9ac0e972d10ed6b9ceaf80f50658855b/.machine_readable/descriptiles/AGENTIC.a2ml) uses another schema (`agentic-config`) and includes `permissions = "read-all"` alongside a tools list containing edit/bash. That is ambiguous for mutation authorization; a tools list is not a write grant.
105+
106+
**Consequence:** canonical-directory-first lookup that ignores legacy or child scopes can lose protections. Finding both layouts is a review case, not permission to delete either one.
107+
108+
### 7. File extensions are not sufficient to select a parser
109+
110+
[EpistemicTypes.jl `rra.a2ml`](https://github.com/hyperpolymath/EpistemicTypes.jl/blob/774d19a897ac665c28dff8eef7ba3caa8ebaacdd/.machine_readable/bot_directives/rra.a2ml) is an S-expression document declaring `dialect "scm"`. Its root-cleaner mode is `putative` (human-review staging), with explicit ignores. The template repo also has this form.
111+
112+
Other directives use TOML-like top-level `allow/deny`, `[bot-directive]`, `[bot]`, `[fleet]`, `[agent-permissions]`, or prose/comments. Filename aliases include `rra`, `robot-repo-automaton` and the referenced `rr_automaton` directory.
113+
114+
**Consequence:** neither renaming SCM to A2ML nor accepting a parseable TOML document guarantees equivalent policy. Unsupported forms must stay unresolved; they must never become implicit permission.
115+
116+
### 8. An enrollment writer is another path to audit before reopening execution
117+
118+
The local `scripts/enroll-hypatia-fleet.sh --apply` unconditionally writes `FLEET-ENROLLMENT.a2ml` for discovered repositories with `.machine_readable`, without first preserving/reconciling an existing enrollment file. Even its default mode writes a local enrollment registry.
119+
120+
It was **not executed** for this census. It remains outside the prior CLI quarantine's documented coverage. Re-enrollment is not a safe substitute for directive discovery. Before enabling automation, protect this writer and all direct fix/script/library paths too.
121+
122+
## What this census does NOT conclude
123+
124+
- A missing observed file means neither consent nor proof that no other instructions exist.
125+
- Metadata date alone does not prove staleness or revoke a denial. Historical exceptions require lifecycle interpretation, not an arbitrary expiry rule.
126+
- Pattern-based candidate discovery is not exhaustive for arbitrary filenames, dynamic config, organization rules, external documentation or private settings.
127+
- Retrieved text was never evaluated as shell/Scheme. No general A2ML/SCM semantic validator was applied.
128+
- GitHub App coverage, branch protection, running hosts, credential scopes and actual enforcement are not established by file presence.
129+
130+
## Next safe implementation milestone
131+
132+
Build a **policy-only resolver with no execution hook**. It should return `blocked`, `requires-owner-review`, or a narrowly scoped proposed decision, with evidence paths and hashes. Start with the concrete cases above as regression fixtures:
133+
134+
1. Preserve all denies/protected paths across canonical, legacy and nested scopes.
135+
2. Treat unavailable, malformed, unknown-schema and conflicting policy as unresolved.
136+
3. Separate bot confidence from permission and distinguish an explicitly configured default from no setting.
137+
4. Bind proposed grants to repo identity, operation, target commit and policy digest.
138+
5. Reject self-modification of policy, require explicit approval where specified, and honor retired exceptions.
139+
6. Keep the current execution quarantine until every mutation entry point is covered and owner-reviewed shadow decisions agree with local intent.
140+
141+
**Do not bulk-generate directives for the 202 repositories without an observed root directive directory.** Review other guidance and request specific enrollment/authorization first.
142+
143+
## Reproduction and tests
144+
145+
```bash
146+
# Read-only GitHub listing and census. Use a NEW external evidence directory.
147+
gh repo list hyperpolymath --limit 2000 \
148+
--json name,isArchived,isPrivate,defaultBranchRef,url > /path/outside/repo/repos.json
149+
bash scripts/audit-directive-census.sh /path/outside/repo/repos.json /path/outside/repo/census
150+
bash scripts/audit-directive-content.sh /path/outside/repo/census
151+
152+
# Offline regression tests; GitHub API is replaced with a fixture executable.
153+
bash tests/directive-census-test.sh
154+
```
155+
156+
Offline tests cover GET-only requests, commit-pinned tree reads, dot-prefixed repositories, unavailable repositories, truncated trees, nested candidates, symlink/submodule recording, blob deduplication and exact CRLF-preserving hash verification. The existing quarantine tests also remain required.

0 commit comments

Comments
 (0)