feat(memory): add dated operating picture tier to memory compile - #9
Open
BohnBawerick wants to merge 5 commits into
Open
feat(memory): add dated operating picture tier to memory compile#9BohnBawerick wants to merge 5 commits into
BohnBawerick wants to merge 5 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Build the dated operating picture tier data/memory/now.md in Firstmate's memory architecture. Requirements: 1. data/memory/now.md is a plain markdown file with a date in front matter holding this shift's pins and ceilings. Ship a committed example template docs/examples/now.md. 2. bin/fm-memory-compile.sh reads now.md only when dated today (matching ISO date in front matter). Precedence under budget pressure: core is never dropped or truncated, then operating picture, then catalog, then hot notes. 3. A stale or undated now.md is never injected into the bundle and reports a notice on emitted output, while absence of now.md is silent and produces byte-identical output to before. 4. Accounted against the startup memory budget via bin/fm-startup-memory-budget-lib.sh. 5. bin/fm-memory-compile.sh --help and header document the new tier. 6. AGENTS.md section 2 mentions the operating picture. 7. Colocated tests in tests/fm-memory-compile.test.sh cover all acceptance criteria.
What Changed
data/memory/now.md) inbin/fm-memory-compile.sh, injecting shift pins and ceilings only when dated today, emitting notices for stale or undated files, and enforcing budget precedence after core and ahead of catalog.docs/examples/now.mdand updatedAGENTS.md,docs/configuration.md, anddocs/documentation-audiences.jsonto document the tier's schema, budget accounting, and lifecycle.tests/fm-memory-compile.test.shfor same-day injection, stale/undated file handling, budget cap precedence, generation isolation, and absent-file parity.Risk Assessment
✅ Low: The dated operating picture tier is cleanly implemented with proper date parsing, home-level vs generation-level scoping, budget precedence, and comprehensive test coverage across all acceptance criteria.
Testing
Executed targeted automated test suites covering compiler behavior, startup memory budget accounting, and documentation inventory classification. Performed end-to-end CLI runs demonstrating that now.md is injected ahead of the catalog when dated today, omitted with a notice when stale or undated, and completely silent when absent. Budget precedence was verified under memory pressure, and the committed docs/examples/now.md template was validated against the compiler. All targeted tests and behavioral checks passed.
Evidence: Operating picture CLI end-to-end verification transcript
Source: Operating picture CLI end-to-end verification transcript
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 9 issues found → auto-fixed (3) ✅
bin/fm-memory-compile.sh:609- The operating picture is read from "$MEMORY/now.md", and $MEMORY is rebound to the active generation directory when data/memory/HEAD exists (lines 175-197). So in any home that uses generations, the documented path data/memory/now.md is never read, and because absence is silent by design there is no notice at all. Verified by hand: home with data/memory/HEAD='gen/1' and data/memory/now.md dated today -> bundle contained neither the body nor any MEMORY_NOTICE (output header read 'COMPILED WORKING MEMORY (data/memory/gen/1)'). This contradicts required criterion 1, 'data/memory/now.md is a plain markdown file with a date in front matter', and criterion 2, 'bin/fm-memory-compile.sh reads now.md only when dated today'. There is precedent for a home-level tier: bin/fm-memory-drop.sh:33-34 always writes $DATA/memory/drop regardless of HEAD. now.md is perishable shift state, not versioned knowledge, so pinning it inside an immutable published generation also means every fm-memory-publish.sh run silently discards the current shift's operating picture. Decide whether now.md should resolve at $DATA/memory/now.md (home-level, like drop/) or stay generation-scoped and be documented as such.bin/fm-memory-compile.sh:565- is_today_date accepts the file date if it equals EITHER the UTC date OR the local date, which widens the validity window past 24 hours anywhere the two differ. Verified by hand: with TZ=America/Los_Angeles (local date 2026-08-19, UTC 2026-08-20), a now.md dated 2026-08-20 was injected, and it will still be injected all through local 2026-08-20 - roughly a 32-hour window. The practical effect west of UTC is that yesterday's shift pins stay live for a large part of today, which is the stale-ceiling failure the tier exists to prevent, and it conflicts with criterion 2's 'only when dated today (matching ISO date in front matter)'. Pick one clock (local is the captain-facing one, and get_today_display already prefers it) and compare against just that.bin/fm-memory-compile.sh:545- parse_now_date assigns both 'date' and 'updated' to the same awk variable, so whichever key appears LAST in the front matter wins. Verified by hand: front matter with 'date: <today>' followed by 'updated: 2020-01-01' produced 'MEMORY_NOTICE: data/memory/now.md is dated 2020-01-01 (not today, 2026-08-20) and is NOT injected' - the documented key was silently overridden and a valid operating picture was dropped. The header contract at lines 51-55 documents only 'date:', so 'updated:' is an undocumented alias with no defined precedence. Make 'date' authoritative (only fall back to 'updated' when no 'date' key was seen), and document the fallback in the OPERATING PICTURE FORMAT block.bin/fm-memory-compile.sh:651- When core+now exceeds the budget, the branch sets both NOW_KEPT=0 and CATALOG_KEPT=0, so a single oversized now.md wipes the catalog and every note even when the catalog would have fit on its own. Reachable whenever a captain writes a verbose shift file: budget=core+catalog+notes but now.md is large -> bundle is core plus a warning only, and the session cannot see that any note exists. The author's own test case 3 asserts this, so it is deliberate, but it works against the reason the header itself gives for catalog precedence ('the catalog is kept ahead of every hot note, because it is the thing that tells the next turn a note exists at all'), and it hands the newest, least-reviewed tier the power to blank the note index. Hot notes already use skip-if-it-does-not-fit rather than stop-the-fill; consider the same for now.md (drop now, keep the catalog), or document this cliff explicitly in the SELECTION AND CAP block.bin/fm-memory-compile.sh:714- The core-alone-over-budget warning still reads 'NOTHING else was: no catalog, no notes' even when a valid, dated-today now.md was also dropped. The updated header at lines 64-66 promises 'no operating picture, no catalog, and no notes' for exactly this case, so the emitted text and the documented contract disagree. Mention the operating picture in the message when NOW_VALID is 1 (keeping the existing wording when it is 0 preserves the byte-identical-when-absent guarantee).bin/fm-memory-compile.sh:725- Two unreachable code paths. (a) Line 725's '(present, empty)' fallback for now.md can never run: NOW_KEPT=1 requires a parsed today-date, which requires front matter, so the file is always non-empty and the '-s' test always passes. (b) Line 766's 'elif NOW_VALID=1 && NOW_KEPT=0 -> STATUS=capped' is redundant, because that state is only produced by the line 651 branch, which also sets CATALOG_KEPT=0, and line 768's existing elif already yields capped. Drop both.bin/fm-memory-compile.sh:559- The nested expansion ${FM_MEMORY_TODAY_OVERRIDE:-${FM_MEMORY_NOW_DATE:-}} is repeated five times across lines 559, 560, 572 and 573. Two env names for one test seam, neither documented in the header, and any future edit has to be applied in five places. Resolve it once into a single variable near the other override handling and document the supported name in the OPERATING PICTURE FORMAT block.docs/examples/now.md:1- The shipped template is only registered in docs/documentation-audiences.json; no test ever feeds it to the compiler, so it can drift out of sync with parse_now_date (for example by gaining an indented or quoted date key) and ship broken. Add a test that copies docs/examples/now.md into a home, reads its front-matter date, and asserts the compiler injects it under that date - that exercises criterion 1 through the real interface rather than by inspecting the file's text.tests/fm-memory-compile.test.sh:655- test_absent_now_md_produces_byte_identical_output_with_no_notice is named for criterion 3's byte-identity guarantee but only asserts the strings 'now.md' and 'operating picture' are absent. The precise regression risk is the MEMORY_ACCOUNTING record gaining a ' now=' field, and the literal 'now.md' does not match 'now='. Add an assertion that accounting_field "$out" now is empty, so the machine-readable line is actually pinned.🔧 Fix: Read now.md home-level, pin local date, keep catalog
4 issues (1 error, 3 infos) still open:
bin/fm-memory-compile.sh:626- The operating picture is read from $DATA/memory/now.md unconditionally, with no dependence on EXPLICIT_MEMORY_DIR. bin/fm-memory-verify.sh:222 verifies a PROPOSED generation by runningfm-memory-compile.sh compile --memory-dir "$GEN_DIR"and reading the resulting MEMORY_ACCOUNTING line, and bin/fm-memory-publish.sh execs that verifier (line 13) with check_budget wired as a blocking gate (bin/fm-memory-verify.sh:750). So the home's perishable, today-dated now.md now counts against a generation's publish gate. Reproduced by hand on the current worktree: home with budget=200, gen/2 core=6 tokens, catalog=35 tokens, and data/memory/now.md dated today at 502 bytes (~168 tokens) ->fm-memory-verify.sh verify gen/2prints 'FAIL budget: core (6 tokens) leaves no room for the catalog within the 200 token budget, so the catalog and every note were dropped and no note is reachable'; deleting now.md and rerunning prints 'PASS budget: ... status=within-budget'. Two defects follow. (a) The gate is non-deterministic: the same generation fails today and passes tomorrow once now.md goes stale, because the tier is dated-today by design. (b) The failure message names the core as the culprit when the core is 6 tokens and now.md is the thing that consumed the budget, so the operator is sent to trim the wrong file. This is a behavior change to a sibling tool, not to the compile path the intent describes; the intent's precedence and accounting requirements are about the startup bundle. Earliest shared boundary that restores the invariant 'verifying a generation depends only on that generation': skip the operating picture when EXPLICIT_MEMORY_DIR is set (the same reasoning that already makescatalogmode target what the caller asked for rather than whatever generation is live, bin/fm-memory-compile.sh:12-18). If the coupling is wanted instead, check_budget must attribute the drop to the operating picture rather than to the core. This challenges a deliberate design decision, so it needs your call.tests/fm-memory-compile.test.sh:833- test_shipped_example_operating_picture_compiles reads the date out of docs/examples/now.md and then feeds that same value back as FM_MEMORY_TODAY_OVERRIDE. Because is_today_date is a plain string equality against TODAY_DATE, any value the template carries becomes 'today' for the duration of the test. If docs/examples/now.md shippeddate: tomorrowordate: 20-08-2026, example_date would be non-empty, the override would match it exactly, and the test would still pass - while a real captain copying the template would get the stale notice on every session. Criterion 1 requires 'a date in front matter' in ISO form. Two cheap additions close the hole: assert example_date matches ^[0-9]{4}-[0-9]{2}-[0-9]{2}$, and add a negative case that compiles the same template with FM_MEMORY_TODAY_OVERRIDE set to a different day and asserts the stale MEMORY_NOTICE fires - that pins the date-gate against the shipped file rather than against a value derived from it.docs/configuration.md:168- docs/examples/now.md is registered in docs/documentation-audiences.json but no prose links it, so a captain reading the memory section never learns a copyable template exists. Every other shipped example follows the opposite convention: docs/configuration.md:331 links docs/examples/crew-dispatch.json and docs/configuration.md:136 and docs/wedge-alarm.md:29 link docs/examples/wedge-alarm. Add the matching sentence after the now.md paragraph, for example: Seedocs/examples/now.mdfor a copyable starting point.bin/fm-memory-compile.sh:741- The change introduces two pairs of near-identical long warning strings that differ only by one clause. Lines 742 and 745 are the same core-alone-over-budget sentence differing only in 'no operating picture, no catalog, no notes' versus 'no catalog, no notes'. Lines 759 and 762 are the same catalog-dropped sentence differing only in whether the operating picture is named in the total and in the remedy. Four copies of two messages means any future wording or remedy edit has to be applied in four places and can silently drift. Each pair collapses to one printf with the varying clause and the varying total resolved into shell variables just above it, which keeps the emitted text identical (including the byte-identical-when-absent guarantee, since the NOW_VALID=0 clause stays empty).🔧 Fix: Scope now.md to named memory dir, dedupe warnings
4 infos still open:
bin/fm-memory-compile.sh:763- The dropped-operating-picture warning always ends with "The catalog and notes below were filled from what remains", but the catalog can be dropped in the same compile. Reproduced by hand on this worktree: home with core=2 tokens, catalog=38 tokens, a today-dated now.md of ~1048 tokens and budget=5 emits, in order, 'MEMORY_BUDGET_WARNING: the core plus operating picture is 1048 estimated tokens ... The catalog and notes below were filled from what remains.' immediately followed by 'MEMORY_BUDGET_WARNING: the core plus catalog is 38 estimated tokens ... so the catalog and every note were dropped.' The first sentence tells the reader a catalog follows when none does. Make the trailing clause conditional on CATALOG_KEPT being 1 (the branch at line 767 already knows), so the message states what actually happened.AGENTS.md:174- AGENTS.md section 2 was updated for the operating picture (criterion 6, satisfied), but the two places that describe what a session start actually injects were not. AGENTS.md:174 still reads 'it carries a standing core, a catalog of every note that exists, and the notes whose triggers matched live fleet work', and docs/configuration.md:178 repeats the same three-part list. Both are now incomplete for any home with a today-dated data/memory/now.md, and AGENTS.md section 3 is the text an agent reads at session start to know what it was given. Add the operating picture to both lists.docs/examples/now.md:7- The shipped operator example's entire body is '- 2 claude + firstmate, 2 grok, 1 opencode, 2 pi, 1 codex last resort', which reads as the author's actual harness fleet rather than a template, and it is committed to a public repo and registered as audience 'operator-example' in docs/documentation-audiences.json. It also does not demonstrate what the header and docs/configuration.md:168 promise the tier is for - 'this shift's pins and ceilings' - so a captain copying it gets a fleet roster, not a pin or a ceiling. A generic body (for example a deploy freeze window and a do-not-touch ceiling) would both remove the private detail and show the intended shape. This is authored content, so it is your call.bin/fm-memory-compile.sh:626- Informational tradeoff, not a defect: bin/fm-memory-verify.sh:222 verifies a proposed generation with 'compile --memory-dir <gen>', which by design now excludes the home-level now.md, while the live session compile adds it and ranks it above the catalog. So a generation can pass the gate's 'catalog is reachable' check (bin/fm-memory-verify.sh:246) and still start a session with no catalog. Reproduced on this worktree: gen/2 core=2, catalog=38, budget=60, home now.md=25 tokens dated today -> 'PASS budget: compiled bundle is 40 estimated tokens <= 60 cap (core=2 catalog=38 status=within-budget)' from the verifier, and 'MEMORY_BUDGET_WARNING: the core plus operating picture plus catalog is 65 estimated tokens against a 60 budget, so the catalog and every note were dropped' from the live compile. This follows directly from the two behaviors you asked for (home-level read for the live compile, generation-scoped read for --memory-dir) combined with intent criterion 2's precedence, and the loud MEMORY_BUDGET_WARNING plus 'now=' in MEMORY_ACCOUNTING make it visible, so no change is required. Noted only so the gate's guarantee is not read as stronger than it now is.🔧 Fix: Condition drop warning remainder and update startup memory docs
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
tests/fm-memory-compile.test.shtests/fm-startup-memory-budget.test.shtests/fm-memory-verify.test.shtests/fm-documentation-audiences.test.shbin/fm-memory-compile.sh --helpEnd-to-end verification ofbin/fm-memory-compile.shwith activenow.md, stalenow.md, undatednow.md, absentnow.md, oversized budget drops, anddocs/examples/now.mdtemplate compilation✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.