From 834ee52deb6f9a0b561252d11bb46f3e8a3c1633 Mon Sep 17 00:00:00 2001 From: Ganesh Viswanath Sambhu <68482658+gvsambhu@users.noreply.github.com> Date: Wed, 12 Aug 2026 11:44:27 +0530 Subject: [PATCH] docs(requirements): re-express against intent sources and record unstated obligations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The published set was produced by transcribing the project plan's requirements section. A transcription inherits its source's structure and cannot notice what was never on it — so it carried the plan's shape, and its gaps, unchanged. This re-expresses the same obligations against the intent sources directly: the plan, the design records' problem statements and decision drivers, the README, and recalled design intent. What the system must do is unchanged; how it is stated, grouped and evidenced is not. Three classes of change. Grouping and form. Nine capability sections replace a split that was release markers wearing capability labels. Front matter the standard expects and the document lacked — purpose, scope, glossary, assumptions, out-of-scope. Compound statements split into a requirement plus acceptance criteria, so obligations that were asserted can now be tested. Release markers moved out of the statements into a status table that doubles as the numeric index. Recorded intent. Fifteen requirements for capabilities that were always intended and never written down — most importantly the three negative guarantees the architecture actually rests on: that no source file is silently dropped, that no source feature is silently dropped from rendered output, and that LLM output must not affect the deterministic artefact. The last is the README's stated thesis and had no requirement in any prior list. Honest status. Every status claim is verified against the code rather than inferred. Five requirements that read as shipped now read as partial, each naming the specific criterion that does not hold. One that read as partial is in fact shipped — its second corpus was deferred by design, not missed. Identifiers are unchanged and unrenumbered; new ones start at FR-40 and NFR-6. The retired NFR-1 keeps its number. Requirements continue to cite no design records, so traceability still runs one way. AI-assistant: Claude Code (Claude Opus 5 via Anthropic) Model: claude-opus-5 --- docs/requirements.md | 610 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 498 insertions(+), 112 deletions(-) diff --git a/docs/requirements.md b/docs/requirements.md index a463aa9..18d31f2 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -6,134 +6,520 @@ requirements it satisfies — so traceability runs from a decision back to its r other way round. Requirements here state *what* the system must do; they do not name the designs that chose *how*. -**Release markers.** Every requirement carries `[v1]` or `[v1.1]`. +**Requirement statements describe the target system.** They are not a changelog of what ships today. +What ships today is recorded once, in the [status table](#status), which is also the numeric index +into this document. -| Marker | Meaning | -|---|---| -| `[v1]` | Shipped in v1 | -| `[v1.1]` | Planned for v1.1 — paths and flags named in these requirements do not exist yet | +**Provenance.** These requirements have governed the project from its start; they were maintained +as project-plan material and first published as this document on 2026-08-05, after v1 shipped. +Publication date is not authorship date. Identifiers are stable and are never reused. A retired requirement keeps its number so existing references continue to resolve. --- -## Functional - -### Core pipeline - -- **FR-1** `[v1]` — Accept any Java/Spring Boot project directory as input (Gradle or Maven). -- **FR-2** `[v1]` — Extract a structured knowledge graph (JSON) from the source, independent of any - render target. Structure is extracted deterministically (FR-10); semantic enrichment is layered on - top of it (FR-11). -- **FR-3** `[v1]` — Render the knowledge graph into idiomatic TypeScript/NestJS source. -- **FR-4** `[v1.1]` — Render the knowledge graph into idiomatic Go source — same graph, different - target. -- **FR-5** `[v1]` — CLI accepts a `--target` flag to select the renderer. -- **FR-6** `[v1]` — All renderers implement a common `Renderer` interface; no target-specific code in - the core pipeline. -- **FR-7** `[v1]` — Run code-quality evaluations per target (compile, coverage, LLM judge) and - produce target-specific scorecards. -- **FR-7a** `[v1]` — Run graph-quality evaluations — structural completeness, relationship - correctness, schema validity, internal consistency, semantic accuracy, reproducibility, and - golden-graph agreement — and produce a graph scorecard. -- **FR-7b** `[v1]` — Maintain documented JSON Schema files for the knowledge graph under - `codeograph/schema/*.schema.json` as the language-neutral source of truth. Every emitted graph - validates against them: enforced at write time through the generated Pydantic models, and - re-checked by the FR-7a schema-validity evaluation. -- **FR-7c** `[v1]` — The deterministic graph (`graph.json`) is regression-tested against committed - golden files under `tests/golden//` for the project's test corpora — a hand-built - edge-case fixture and a pinned real Spring project. CI diffs every run byte-equal in canonical - form. Golden refreshes are deliberate (`--update-goldens`), and each refresh records its diff - category: update-golden, fix-bug, or intentional-change. The committed example run (FR-9) ships - its `graph.json` as illustrative output, not as a byte-equal regression golden. -- **FR-8** `[v1]` — Support provider switching by environment variable. The v1 provider set is - `anthropic | openrouter | openai_compatible`; `ollama` and `bedrock` are `[v1.1]`. The - `openai_compatible` provider reaches any API-key, OpenAI-compatible endpoint through a +## 1. Purpose + +Codeograph reads a Java/Spring Boot codebase and produces two things: a **deterministic knowledge +graph** of what is in it — structure, relationships, and named complexity metrics — and, driven from +that graph, **idiomatic source in another language stack**. An LLM layer sits on top to explain and +translate; it never becomes the authority on what the code *is*. + +This document defines what the system must do for that to be true, and what it must promise never to +do. + +## 2. Scope + +**In scope.** Reading a Java/Spring Boot codebase supplied as a local directory, a git URL, or a zip +archive; extracting structure, framework semantics, and complexity deterministically; enriching that with +LLM-derived semantics kept separable from the deterministic output; rendering a selected subset into a target stack; +measuring the quality of both the graph and the rendered output; and the operational surface that +makes a run reproducible, inspectable, and bounded in cost. + +**Out of scope.** Building, running, testing, or deploying either the source project or the rendered +output. Runtime or dynamic analysis. Non-JVM source languages. Security scanning, architecture +governance, and same-language refactoring — mature tools already cover those. Migrating data, +infrastructure, or build pipelines. Any claim that rendered output is production-ready without +review. + +## 3. Definitions + +| Term | Meaning in this document | +|---|---| +| **Corpus** | The set of source files in scope for one run, after acquisition and discovery. Not the same as "the repository" — excluded and non-source files are not part of the corpus. | +| **Run** | One end-to-end invocation of the tool against one corpus, producing one output directory. | +| **Artefact** | A file the tool writes into the output directory as part of its contract with consumers (graph, annotations, manifest, logs, telemetry, scorecards, rendered source). | +| **Deterministic artefact** | An artefact whose bytes are a pure function of the corpus and the tool version — reproducible across runs and machines. | +| **Knowledge graph** | The deterministic property graph of nodes (classes, methods) and edges (calls, injection, inheritance, persistence relations) carrying structural facts and complexity metrics. | +| **Enrichment** | LLM-derived semantic output about the graph — role, intent, summaries, migration hazards, corpus-level synthesis. Always advisory, and always separable from the deterministic output. | +| **Domain** | A deterministically-computed grouping of classes used to partition the corpus for rendering. It is derived from the graph, not assigned by an LLM. | +| **Render target** | A language/framework stack the graph can be rendered into, supplied by a renderer implementing the common interface. | +| **Idiomatic** | Output a competent engineer in the target stack would recognise as written for that stack — using its own framework conventions — rather than transliterated Java. | +| **Scorecard** | A machine-readable JSON record of quality checks, each with an id, a result, a threshold, and the rationale for that threshold. | +| **Golden** | A committed known-good artefact for a pinned input, compared byte-for-byte on every run to catch unintended drift. | +| **Canonical form** | The single byte-level serialisation rule for an artefact (key ordering, separators, encoding, line ending) that makes byte-comparison and hashing meaningful. | + +## 4. Assumptions + +1. The user runs the tool locally against source they are entitled to read. +2. A JDK is available for full-fidelity parsing. (What the tool does when one is absent is an + obligation, not an assumption — see FR-31.) +3. `git` is available when a git URL is supplied as input. +4. The target project's own build tooling is **not** invoked and need not be installed. +5. LLM access requires the user's own provider credentials, supplied through the environment; any + run may be performed with the LLM layer switched off entirely. +6. Rendered output is a reviewable starting point, not a finished port. + +--- + +## 5. Functional requirements + +### 5.1 Source intake + +#### FR-1 · Accept a Java/Spring Boot project regardless of its build system +The tool works on any reasonably-shaped Spring Boot project without per-project configuration. +- **AC-1** Maven projects are accepted. +- **AC-2** Gradle projects are accepted. +- **AC-3** The detected build system is recorded in the run's output as Maven or Gradle. It is never + recorded as unknown. + +#### FR-29 · Accept the source in any of the three forms a codebase is handed over in +Input handling covers the real handover paths, not just the local checkout. +- **AC-1** A local directory path is accepted. +- **AC-2** A git URL is accepted and the repository is acquired. +- **AC-3** A `.zip` archive is accepted and extracted. +- **AC-4** Archive extraction is bounded and path-checked (FR-24 AC-1, AC-2). + +#### FR-30 · Discover the in-scope sources of the project +Turning "a project" into "a known set of files" is a defined, inspectable step. +- **AC-1** Multi-module projects are enumerated, and every source file is labelled with the module it + belongs to. +- **AC-2** Files excluded by the project's own ignore rules are not in the corpus, so custom output + and generated-source directories are skipped without per-project configuration. +- **AC-3** Files are categorised by role, so non-source files never reach the parser. + +### 5.2 Deterministic extraction + +#### FR-10 · Structural and syntactic facts are extracted deterministically, not by an LLM +Classes, methods, annotations, imports, and dependencies are syntactic; they must come out identical +on every run. +- **AC-1** No LLM call participates in producing any structural fact. +- **AC-2** The same corpus yields the same structural facts on re-run (NFR-6). + +#### FR-31 · No source file is ever silently dropped +**Negative guarantee.** Every file in the corpus is represented in the graph, whatever went wrong +while reading it. +- **AC-1** A file that cannot be fully parsed still yields a node, marked with the confidence of the + extraction that produced it. +- **AC-2** A file that yields no usable structure at all still yields an identifiable node, marked + lowest-confidence. +- **AC-3** A failure of the parsing machinery degrades the affected files and reports it at run level; + it never silently aborts the run. +- **AC-4** Every degraded file is recorded in the run's output with the reason. + +#### FR-32 · Framework semantics are interpreted, not merely captured +The graph carries Spring's meaning, not just its annotation text. +- **AC-1** Every annotation present in source is captured on the node it applies to. +- **AC-2** A documented subset is additionally *interpreted* — lifted to a structured field, an edge, + or a hazard tag. The membership criterion is documented: an annotation is interpreted when it lifts + to a structured graph field or edge. +- **AC-3** Members generated by compile-time code generation (e.g. Lombok) are synthesised so the + graph reflects the compiled shape of the class, not only its written text. + +#### FR-40 · An unresolvable reference is recorded as unresolved, never dropped +**Negative guarantee.** A consumer reading the graph can tell "there is no call here" apart from +"there is a call we could not trace." +- **AC-1** A reference the tool cannot resolve to a known target is still emitted, marked unresolved. +- **AC-2** An absent edge therefore means the reference is absent from the source — never that + resolution failed. +- **AC-3** Conditions that degrade resolution across the run are reported to the user, with the + remediation where one exists. + +#### FR-12 · The complexity scorecard uses named industry-standard metrics with numeric thresholds +- **AC-1** Each metric is named, and its computation method cites a published source. +- **AC-2** Every reference threshold value cites a published source — a paper or a published rule + reference. No threshold is chosen from memory. +- **AC-3** Values are numeric; no subjective high/medium/low bucketing is emitted in place of a value. +- **AC-4** A metric that cannot be computed for a node is null, never guessed. + +### 5.3 Semantic enrichment + +#### FR-11 · Semantic facts are extracted by an LLM operating on the deterministic output +Service intent, domain boundaries, and migration hints are inferred; they are inferred *from* the +extracted structure, not from raw source in place of it. + +#### FR-35 · Enrichment degrades; individual failures do not fail the run +**Negative guarantee.** A provider outage, rate limit, or oversized input costs quality rather than +the run — up to a defined bound (AC-3). +- **AC-1** A per-call failure is recorded and the run proceeds; the affected node keeps its + deterministic facts and is marked as un-enriched. +- **AC-2** An input too large for quality extraction is reduced rather than dropped, and the + reduction is recorded on the node. +- **AC-3** There is a defined threshold of enrichment failure past which the run fails loudly rather + than emitting a substantially unenriched result. +- **AC-4** The full pipeline can be run with LLM passes disabled, producing the deterministic + artefacts alone. + +### 5.4 Output artefacts and their contracts + +#### FR-34 · LLM output must not affect the deterministic artefact +**Negative guarantee — the central architectural property.** The knowledge graph is a function of the +corpus and the tool alone. Nothing an LLM returns, and nothing about whether an LLM was reachable, +may change a single byte of it. +- **AC-1** No LLM-derived value is written into the deterministic graph artefact. +- **AC-2** No LLM *call outcome* (success, failure, cache hit, skip) is recorded in the deterministic + graph artefact. +- **AC-3** A run with enrichment disabled and a run with enrichment enabled produce byte-identical + deterministic graphs for the same corpus. +- **AC-4** The deterministic output is separable: a consumer can obtain it on its own, byte-identically, + with enrichment joined to it by node identity at consumption time. + +#### FR-2 · Emit a structured knowledge graph, independent of any render target +The graph is the product, not an intermediate. Structure is extracted deterministically (FR-10); +semantic enrichment is layered on top of it (FR-11). +- **AC-1** The graph is a language-neutral property graph of nodes and edges. +- **AC-2** It carries no target-language concepts. +- **AC-3** Every class in the corpus is present in the graph, whatever the rendering budget (FR-13). + +#### FR-7b · The graph schema is a published, machine-readable contract +Documented JSON Schema files are the language-neutral source of truth for the emitted artefacts. +- **AC-1** Every emitted graph validates against the published schema at write time. +- **AC-2** Validity is re-checked independently by the graph scorecard (FR-7a). +- **AC-3** The schema is published as a committed artefact, so an external consumer can validate + output without importing the producing package. + +#### FR-20 · Every run writes a manifest and structured logs +- **AC-1** A run manifest is written into the output directory as the entry point to the run. +- **AC-2** Structured, machine-readable logs are written into the output directory for the same run. +- **AC-3** A run correlation identifier joins manifest, logs, and telemetry. +- **AC-4** Human-readable console output and the structured log file are emitted together; verbosity + is controllable at invocation. + +#### FR-33 · The manifest is an integrity contract over the run's artefacts +- **AC-1** The manifest indexes every artefact the run produced. +- **AC-2** Each indexed artefact carries a SHA-256 of its committed bytes. +- **AC-3** A missing artefact is explicit — an absent hash never doubles as "this stage was skipped". +- **AC-4** A consumer can detect any post-run modification of an artefact from the manifest alone. + +#### FR-28 · A determinism contract documents per-field determinism classification for the graph schema + +### 5.5 Rendering + +#### FR-3 · Render the knowledge graph into idiomatic TypeScript/NestJS source +- **AC-1** The rendered project runs from a clean checkout under its default configuration, without + manual repair. +- **AC-2** Each source class maps to an identifiable unit of the output, so source and output can be + compared side by side. + +#### FR-4 · Render the knowledge graph into idiomatic Go source +Same graph, different target — the proof that the graph is target-independent. + +#### FR-5 · The render target is selectable at invocation, without a code change +- **AC-1** A `--target` flag selects the renderer. +- **AC-2** The set of available targets is discoverable at runtime. +- **AC-3** An unknown target fails before pipeline work begins. + +#### FR-6 · All renderers implement a common interface; no target-specific code sits in the core pipeline + +#### FR-13 · Rendering output is capped per domain +Bounds token spend and keeps runs reproducible. +- **AC-1** The default cap is 3 classes per domain. +- **AC-2** A value of `0` disables the cap entirely. +- **AC-3** Selection at the cap is deterministic and documented (FR-14). +- **AC-4** The user is told at run end that the cap bound, and for which domains, so a capped output + is never mistaken for a full run. +- **Interface** `--max-classes-per-domain N` + +#### FR-14 · Selection under the cap is deterministic and documented +- **AC-1** The same corpus and cap select the same classes on every run. +- **AC-2** The ordering rule is documented. Where it uses thresholds, each cites a published source. +- **AC-3** The selection is auditable — what was selected, what was skipped, and by which rule. +- **AC-4** The knowledge graph always covers every class; only rendering is capped. +- **AC-5** The partition of the corpus into domains is computed deterministically from the graph. + +#### FR-36 · No source feature is ever silently dropped from rendered output +**Negative guarantee.** Anything the renderer cannot translate faithfully is visible in the output. +- **AC-1** An untranslatable feature surfaces as a reviewable placeholder carrying the original + intent. +- **AC-2** Security-relevant features default to refusal, so a rendered endpoint is never silently + less protected than its source. +- **AC-3** The encounter behaviour is configurable, and the permissive setting warns loudly. +- **AC-4** The set of features v1 translates, defers, and refuses is documented as a single matrix. + +#### NFR-5 · Adding a target language requires no change to the analyzer or the core pipeline +- **AC-1** A new target is a new renderer package, a compile-check entry, and nothing else. +- **AC-2** Renderer-authoring documentation exists for a contributor adding one: a short section in + the contributor guide, with the full guide landing alongside the second renderer. + +### 5.6 Evaluation + +#### FR-7 · Run code-quality evaluations per target and produce target-specific scorecards +- **AC-1** The rendered output is compile-checked with the target stack's own toolchain. +- **AC-2** Feature-translation coverage is measured against the documented feature matrix (FR-36 AC-4). +- **AC-3** An LLM judge scores output quality. +- **AC-4** A check that cannot run records an explicit skip with a reason; a skip is never reported + as a pass. + +#### FR-7a · Run graph-quality evaluations and produce a graph scorecard +Checks: structural completeness, relationship correctness, schema validity, internal consistency, +semantic accuracy, reproducibility, and golden-graph agreement. +- **AC-1** Each check records a result, the threshold applied, and the rationale for that threshold. +- **AC-2** Every check is deterministic, or explicitly skipped with a reason. +- **AC-3** Scorecards are machine-readable artefacts a consumer can locate from the run's manifest. + +#### FR-41 · Graph correctness is measured against a human-validated expectation +Well-formedness checks cannot detect a fact the extractor never produced. Measuring accuracy requires +a statement of what the graph *should* contain, derived from reading the source rather than from the +tool's own output. +- **AC-1** For at least one corpus, an expectation of the graph's contents is authored from the source + and confirmed by a person, independently of what the tool emits. +- **AC-2** The expectation is never regenerated from tool output — capture would record the tool's own + errors as correct. +- **AC-3** The expectation may be partial. Which entries have been confirmed is recorded, and accuracy + is reported only over the confirmed set. +- **AC-4** Runs are compared against it, and every difference is classified as improvement, regression, + or intentional change. +- **AC-5** The expectation covers both what deterministic extraction should find and what the semantic + layer should conclude. The semantic comparison is tolerance-based, not exact. + +> This is distinct from FR-7c. FR-7c detects **drift** — whether output changed between runs — using a +> baseline captured from the tool. FR-41 measures **correctness** — whether output is right — and +> cannot use a captured baseline for that purpose. A system can be perfectly stable and consistently +> wrong; only FR-41 detects it. + +#### FR-18 · A scorecard over LLM-derived output records the exact model version that produced it +Exact version strings, never family names — so a score that moves can be attributed to the tool or to +the model. +- **AC-1** Where the artefact being scored was produced by an LLM, the scorecard records the model + version and the prompt version that produced it. +- **AC-2** Where the artefact was produced deterministically, no model provenance is recorded, and its + absence is not a gap. + +#### FR-9 · Ship at least one complete example run per target, with committed output and scorecards +- **AC-1** The committed example includes the run's own output artefacts. +- **AC-2** It includes the graph scorecard and the target scorecard. +- **AC-3** For a render target, it includes the rendered source. +- **AC-4** Example output is illustrative, not a byte-equal regression golden (FR-7c). + +#### FR-25 · An LLM-judge calibration suite discriminates known-good from known-bad output at documented thresholds, committed as a calibration record + +#### NFR-3 · Evaluation results are reproducible +Seed is tracked per target; where the scored artefact was LLM-derived, so are the model version and +prompt version that produced it (FR-18) — so a past score can be re-derived. + +### 5.7 LLM platform control + +#### FR-8 · Provider and model are both selectable, and neither is allowlisted +Two independent axes: which provider runs the call, and which model it runs. +- **AC-1** The provider is selectable by environment variable. The v1 provider set is + `anthropic | openrouter | openai_compatible`; `ollama` and `bedrock` follow. +- **AC-2** The `openai_compatible` provider reaches any API-key, OpenAI-compatible endpoint through a configurable base URL, with no coded vendor allowlist; `openrouter` is its OpenRouter-URL preset. - All providers accept any model id as free-form pass-through — no project-blessed allowlist. -- **FR-9** `[v1]` — Include at least one complete example run per target, with committed output and - evaluation scorecards. -- **FR-10** `[v1]` — Structural and syntactic facts — classes, methods, annotations, imports, - dependencies — are extracted deterministically from the source, not by an LLM. -- **FR-11** `[v1]` — Semantic facts — service intent, domain boundaries, migration hints — are - extracted by an LLM operating on the deterministic output. -- **FR-12** `[v1]` — The complexity scorecard uses named industry-standard metrics with numeric - thresholds. No subjective high/medium/low buckets. -- **FR-13** `[v1]` — CLI accepts `--max-classes-per-domain N` (default 3) to cap rendering output - per domain, bounding token spend and keeping runs reproducible. A value of `0` disables the cap. -- **FR-14** `[v1]` — When the cap is reached, rendering selects classes deterministically, ordered by - a documented rule, so runs are reproducible. The knowledge graph always covers every class — only - rendering is capped. - -### Quality infrastructure - -- **FR-15** `[v1]` — All prompts live as versioned files under `codeograph/prompts/`; scorecards - record the prompt filenames and versions used. -- **FR-16** `[v1]` — Every LLM call emits telemetry: `run_id`, provider, model, prompt file, prompt - version, input and output tokens, cost estimate, latency, and timestamp. Per-call records are - written to a sidecar JSONL; aggregate statistics roll into the run manifest. `run_id` is the - correlation key joining manifest, logs, and telemetry. -- **FR-17** `[v1]` — LLM responses are cached content-addressed on provider, model, prompt version, - rendered prompt, and input hash. Cache hits are recorded in telemetry. -- **FR-18** `[v1]` — Every scorecard records exact model version strings, not family names. -- **FR-19** `[v1]` — A `pytest` suite at `tests/` covers unit and integration tests. CI blocks merge - on failure; unit-test coverage target is 80% or above. -- **FR-20** `[v1]` — Every tool invocation writes a run manifest (`manifest.json`) and structured - logs (`logs.jsonl`) into the output directory, alongside the run's other artefacts. -- **FR-21** `[v1]` — Secret scanning runs in CI on every push and pull request; merges are blocked on - detection. -- **FR-27** `[v1]` — Output-path safety: all run artefacts are written inside the output directory; - an output directory that is the working directory, or an ancestor of it, is rejected; a non-empty - output directory is never silently overwritten — `--force` is required. - -### Quality hardening - -- **FR-22** `[v1.1]` — CLI supports cost estimation before a run (`--dry-run`) and a hard spend - ceiling (`--max-cost-usd`). *v1 ships an interim mechanism instead: call and token ceilings with a - confirmation gate (`--max-llm-calls`, `--max-tokens-total`, `--llm-call-confirm-threshold`).* -- **FR-23** `[v1.1]` — Snapshot tests lock byte-stable rendered output for a fixed input and cached - responses; unexplained drift fails CI. -- **FR-24** `[v1.1]` — Negative tests assert graceful failure with specific error messages for at - least six malformed-input classes. The baseline classes: - 1. **Zip bomb** — extracted size exceeds the configured cap; abort with a size-limit error before - extraction completes. - 2. **Zip path traversal** — an archive entry resolves outside the extraction root; reject the - archive. - 3. **Corrupt or malformed archive** — input is not a readable archive; return a clear error. - 4. **Acquisition failure** — the source cannot be fetched, or the URL is unreachable or - credential-gated; return an actionable error naming the cause. - 5. **Empty corpus** — no `.java` files remain in scope after discovery; fail loudly with a - user-readable message and a non-zero exit, never an empty-graph pass-through. - 6. **No recognised build file** — neither a Maven nor a Gradle build file is present; reject the - corpus with a clear error rather than recording an unknown build system. -- **FR-25** `[v1.1]` — An LLM-judge calibration suite discriminates known-good from known-bad output - at documented thresholds, committed as a calibration record. -- **FR-26** `[v1.1]` — Prompt-injection mitigation: source is wrapped in data tags with explicit - instruction guards, escaped for prompt-breaking sequences, and validated by a negative test. -- **FR-28** `[v1.1]` — A determinism contract documents per-field determinism classification for the - graph schema. +- **AC-3** Any model id is accepted as free-form pass-through — no project-blessed allowlist. +- **AC-4** Per-pipeline-stage model overrides are available, each falling back to the single + configured default. + +#### FR-15 · Prompts are versioned artefacts, not inline strings +- **AC-1** Each prompt version is a distinct, reviewable file; editing one in place is not how a new + version is made. +- **AC-2** The production default for a prompt is declared explicitly, so promotion is a visible diff. +- **AC-3** A scorecard over LLM-derived output records the prompt identity and version that produced + it (FR-18). +- **AC-4** A prompt declares the inputs it requires, and a missing or unknown input fails before the + call is made. + +#### FR-16 · Every LLM call emits telemetry +- **AC-1** Each call records: run id, provider, model, prompt identity, prompt version, input tokens, + output tokens, cost estimate, latency, and timestamp. +- **AC-2** Per-call records are written to a sidecar JSONL artefact. +- **AC-3** Aggregate statistics roll into the run manifest. +- **AC-4** The run id is the correlation key joining manifest, logs, and telemetry. + +#### FR-17 · LLM responses are cached content-addressed +- **AC-1** The cache key covers everything that would change the response — provider, model, prompt + version, rendered prompt, and input hash — and nothing that would not. +- **AC-2** A stale entry is never served after any of those change. +- **AC-3** Cache hits are recorded in telemetry. + +#### FR-37 · The response cache is inspectable and user-managed +- **AC-1** The user can report cache size, entry count, and hit behaviour. +- **AC-2** The user can purge selectively — by age, by model, by prompt version, or entirely. +- **AC-3** Nothing is evicted by surprise; growth is surfaced as an advisory, and reproducibility of + a past run is never broken without the user acting. + +#### FR-39 · LLM spend is bounded before a run starts and while it runs +No first run can launch an unbounded paid fan-out. +- **AC-1** A pre-flight estimate of call count and cost is shown before any paid work begins, and is + labelled as an estimate from a dated table, not a quote. +- **AC-2** Above a configurable call-count threshold, an interactive run asks for confirmation. +- **AC-3** A non-interactive run never hangs on that prompt and never silently proceeds past it. +- **AC-4** Optional hard ceilings on total calls and total tokens abort the run mid-flight, reporting + what was completed. +- **Interface** `--max-llm-calls`, `--max-tokens-total`, `--llm-call-confirm-threshold`, `--yes`, + `--non-interactive` + +#### FR-22 · Cost is estimated and capped in dollars +The permanent budget surface, superseding the call-and-token floor of FR-39. +- **AC-1** A dry run estimates cost without making paid calls. +- **AC-2** A hard spend ceiling stops the run on projected or accumulated spend. +- **Interface** `--dry-run`, `--max-cost-usd` + +#### NFR-2 · A defined token-utilization strategy governs LLM input construction +- **AC-1** Repeated prompt prefixes are not re-billed per call. +- **AC-2** A token and cost scorecard surfaces spend per run. + +### 5.8 Configuration and run safety + +#### FR-38 · Configuration is layered, with an explicit and predictable precedence +This is the falsifiable form of "the tool is configurable". +- **AC-1** Settings resolve in a documented order: invocation flags, then environment, then local + environment file, then project config file, then built-in defaults. +- **AC-2** Configuration is validated at startup; a misconfiguration fails before work begins, not + mid-run. +- **AC-3** Secrets are read only from the environment and the local environment file, never from the + committed config file. +- **AC-4** A committed example environment file documents every credential a contributor must set. + +#### FR-27 · Output-path safety +- **AC-1** All run artefacts are written inside the output directory. +- **AC-2** An output directory that is the working directory, or an ancestor of it, is rejected. +- **AC-3** A non-empty output directory is never silently overwritten — `--force` is required. + +#### FR-26 · Prompt-injection mitigation +Source is wrapped in data tags with explicit instruction guards, escaped for prompt-breaking +sequences, and validated by a negative test. + +### 5.9 Engineering assurance + +#### FR-19 · A test suite covers unit and integration behaviour, and CI blocks merge on failure +- **AC-1** Unit and integration layers are separated and independently runnable. +- **AC-2** Unit-test coverage is at or above 80%, enforced as a hard gate. +- **AC-3** No test requires a live LLM provider or network access. + +#### FR-7c · The deterministic graph is regression-tested against committed goldens +- **AC-1** A golden is committed for a hand-built edge-case fixture corpus. +- **AC-1b** A golden is committed for a pinned real Spring project. *(v1.1 — the harness ships with + the fixture corpus alone; the real-project corpus follows.)* +- **AC-2** CI diffs every run byte-equal in canonical form. +- **AC-3** A golden refresh is deliberate (`--update-goldens`), and each refresh records its diff + category: update-golden, fix-bug, or intentional-change. + +#### FR-23 · Snapshot tests lock byte-stable rendered output for a fixed input and cached responses; unexplained drift fails CI + +#### FR-24 · Negative tests assert graceful failure with specific error messages +At least six malformed-input classes, each failing loudly and actionably: +1. **Zip bomb** — extracted size exceeds the configured cap; abort with a size-limit error before + extraction completes. +2. **Zip path traversal** — an archive entry resolves outside the extraction root; reject the archive. +3. **Corrupt or malformed archive** — input is not a readable archive; return a clear error. +4. **Acquisition failure** — the source cannot be fetched, or the URL is unreachable or + credential-gated; return an actionable error naming the cause. +5. **Empty corpus** — no `.java` files remain in scope after discovery; fail loudly with a + user-readable message and a non-zero exit, never an empty-graph pass-through. +6. **No recognised build file** — neither a Maven nor a Gradle build file is present; reject the + corpus with a clear error rather than recording an unknown build system. + +#### FR-21 · Secret scanning runs in CI on every push and pull request; merges are blocked on detection + +#### NFR-4 · CI runs lint, unit tests, and the evaluation path on every push +- **AC-1** Lint and unit tests gate merge. +- **AC-2** The evaluation path runs per committed example corpus and its scorecards are retained. +- **AC-3** The evaluation path exercises each shipped render target's own toolchain. + +#### NFR-6 · The deterministic artefacts are byte-reproducible +Re-running the tool on the same corpus with the same version, within the declared execution +environment (AC-4), produces byte-identical deterministic artefacts. +- **AC-1** A single canonical serialisation rule governs those artefacts — key ordering, separators, + encoding, and line ending. +- **AC-2** The canonical bytes are what is hashed for integrity (FR-33 AC-2), so one mechanism serves + both comparison and integrity. +- **AC-3** No machine-specific value — absolute path, timestamp, locale, or iteration order — reaches + a deterministic artefact. +- **AC-4** The execution environment under which byte-equality holds is declared and pinned. Drift + from it is detected and stops the comparison, rather than being reported as a difference in output. + +#### NFR-7 · Artefact schema versions are governed independently of the application version +Consumers pin the format, not the tool. +- **AC-1** Every emitted artefact's schema version is declared in the run's manifest, independently + per artefact. +- **AC-2** Artefact schema versions and the application version are separate version lines; neither + implies the other. +- **AC-3** Within a major schema line, evolution is additive, so an existing consumer keeps working. +- **AC-4** A breaking change to an artefact's shape requires a major bump of that artefact's schema + version. --- -## Non-functional +## 6. Retired identifiers - **NFR-1** — **Retired.** No longer tracked as a requirement. The number is retained so existing references resolve. -- **NFR-2** `[v1]` — A defined token-utilization strategy governs LLM input construction; v1 ships - prefix caching. `[v1.1]` A token and cost scorecard surfaces spend per run. -- **NFR-3** `[v1]` — Evaluation results are reproducible: seed, model version, and prompt version are - tracked per target. -- **NFR-4** `[v1]` — CI runs lint, unit tests, and the TypeScript evaluation path on every push. - `[v1.1]` The Go evaluation path lands with the Go renderer. -- **NFR-5** `[v1]` — Adding a new target language requires no changes to the analyzer or the core - pipeline — only a new renderer, a compile-check entry, and renderer-authoring documentation. v1 - ships a short "adding a renderer" section in `CONTRIBUTING.md`. `[v1.1]` The full renderer guide - lands with the second renderer, which is what makes the pattern real. --- -## Revision history +## 7. Status + +Requirement statements above describe the target system. This table records what ships today. +`shipped` · `partial` · `planned` · `retired`. + +| ID | Status | Target | Note | +|---|---|---|---| +| FR-1 | shipped | v1 | Maven and Gradle both accepted. Gradle classpath resolution falls back to source-only, so method-call resolution is lower fidelity | +| FR-2 | shipped | v1 | | +| FR-3 | partial | v1 | AC-2 holds — per-class render granularity plus a deterministic project scaffold; full method bodies, not skeletons. AC-1 is not yet verified — the compile check runs the type-checker only, and no check installs or starts the rendered app | +| FR-4 | planned | v1.1 | | +| FR-5 | shipped | v1 | `--target` on `render`; `--list-targets` enumerates | +| FR-6 | shipped | v1 | Registry-based; one renderer registered | +| FR-7 | partial | v1 | Compile check runs the target toolchain. AC-2 skips pending the render-time producer for its input. AC-3 llm_judge deferred | +| FR-7a | partial | v1 | Six of seven checks run; `semantic_accuracy` returns an explicit skip pending judge calibration. Separately, `structural_completeness` runs but its ground-truth source is an open question — a completeness ratio can only count declarations the extractor produced, so a declaration it never saw is absent from both sides. FR-41 is the intended oracle | +| FR-7b | shipped | v1 | Graph and annotation schemas published; manifest schema published and CI-gated against drift | +| FR-7c | shipped | v1 | AC-1 and AC-2 hold — the hand-built fixture corpus golden is committed and CI-gated. **AC-1b is correctly deferred**, not missing: the real-project corpus and its golden were moved to v1.1 by design, and the fixture golden satisfies the v1 drift guardrail | +| FR-8 | shipped | v1 | `anthropic`, `openrouter`, `openai_compatible` live; `ollama` and `bedrock` wired in the resolver but raise `NotImplementedError`. Per-stage overrides present; no curated differential mapping | +| FR-9 | partial | v1 | Two committed example corpora ship a graph scorecard each. No committed run output, target scorecard, or rendered source | +| FR-10 | shipped | v1 | | +| FR-11 | shipped | v1 | | +| FR-12 | shipped | v1 | Cyclomatic, cognitive, WMC, CBO, LCOM4, MLOC — each cited; null where not computable | +| FR-13 | partial | v1 | Default 3 ships. AC-2 not yet met — `0` is currently rejected rather than disabling the cap | +| FR-14 | shipped | v1 | Stratified selection with cited thresholds; audit fields persisted | +| FR-15 | partial | v1 | Versioned prompt files with content-hash pinning ship. AC-3 not yet met — the scorecard `prompt_id` field is present but unpopulated | +| FR-16 | partial | v1 | All fields emitted to the sidecar JSONL and aggregated into the manifest. Cost estimate is hardcoded `0.0` — the field exists, the value is never computed | +| FR-17 | shipped | v1 | SQLite content-addressed store; key covers all response-affecting inputs | +| FR-18 | partial | v1 | AC-2 holds — the graph scorecard records no model provenance, correctly, since its subject is deterministic. **AC-1 unmet**: the code scorecard grades LLM-rendered source and its `model_version` / `prompt_id` fields, present in both the model and the published schema, are never populated | +| FR-19 | shipped | v1 | `--cov-fail-under=80` enforced in CI; mock provider, no live calls | +| FR-20 | shipped | v1 | `manifest.json` + `logs.jsonl` in the output directory; `run_id` correlates; dual-channel emission with `-v`/`-q`/`--log-level` | +| FR-21 | shipped | v1 | Exact-pinned gitleaks in CI with a parity check and a nightly full-history scan; no admin bypass | +| FR-22 | planned | v1.1 | Superseded by FR-39 for v1; FR-39 is removed when this lands | +| FR-23 | planned | v1.1 | | +| FR-24 | planned | v1.1 | | +| FR-25 | planned | v1.1 | | +| FR-26 | planned | v1.1 | | +| FR-27 | shipped | v1 | cwd-or-ancestor rejection and non-empty `--force` gate both verified | +| FR-28 | planned | v1.1 | | +| FR-29 | shipped | v1 | Local, git, and zip acquirers all present | +| FR-30 | shipped | v1 | Multi-module enumeration with module labelling; ignore-rule-driven exclusion | +| FR-31 | shipped | v1 | AST → regex → name-only stub ladder; degraded files recorded | +| FR-32 | shipped | v1 | Interpretation runs inside the parser; interpreted set documented with its membership criterion | +| FR-33 | shipped | v1 | SHA-256 required on every present pointer; skipped LLM passes signalled by an explicit flag, not a null hash | +| FR-34 | shipped | v1 | Enforced as a file boundary — the deterministic graph carries no LLM-derived value and no call outcome | +| FR-35 | shipped | v1 | Per-call skip, oversized-input reduction, configurable abort ratio, and a full LLM-off mode | +| FR-36 | shipped | v1 | Placeholder/refuse policies configurable, security defaulting to refuse. The bundled preset flags are not on the CLI; policies are set through configuration | +| FR-37 | shipped | v1 | `cache stats`, `cache purge` (dry-run by default), `cache report` | +| FR-38 | shipped | v1 | Precedence documented in the committed config file and enforced by the settings layer | +| FR-39 | shipped | v1 | Pre-flight estimate from a dated local price table, TTY-aware confirmation gate, and mid-run call/token ceilings | +| NFR-1 | retired | — | Number retained so existing references resolve | +| NFR-2 | partial | v1 | Prefix caching ships. AC-2 token and cost scorecard not shipped, and blocked behind FR-16's uncomputed cost | +| NFR-3 | partial | v1 | Scorecard carries `seed`. Model and prompt provenance for LLM-derived output is unpopulated — same root cause as FR-18 AC-1 and FR-15 AC-3 | +| NFR-4 | partial | v1 | Lint, unit tests, and a per-corpus eval job all run. AC-3 not yet met — the eval job runs the deterministic path and does not invoke the target toolchain | +| NFR-5 | partial | v1 | The architecture holds — a renderer is a self-contained package behind a registry. AC-2 not yet met — the renderer-authoring section is not yet written | +| FR-40 | shipped | v1 | The graph schema carries a `calls_unresolved` edge kind, so an unresolvable reference is emitted rather than dropped. AC-3's degraded-resolution reporting is decided (remediation hint on an unpopulated local Maven repository) but not separately verified here | +| FR-41 | planned | v1.1 | Not shipped in any form. FR-7c's harness detects drift against a captured baseline, which cannot serve as a correctness oracle. The two accuracy checks that need this expectation are `structural_completeness` (its ground-truth source is an open question) and `semantic_accuracy` (skipped, deferred) | +| NFR-6 | shipped | v1 | Canonical serialisation in the writer; the same bytes feed the manifest hash and the golden assertion | +| NFR-7 | shipped | v1 | Graph and manifest schema versions are independent of the application version; manifest is `2.0.0`, strict-additive within `2.x` | + +--- + +## 8. Revision history These requirements were first drafted before implementation began, and were revised during design review as decisions sharpened or rescoped them. **This section closes at this document's first @@ -146,7 +532,7 @@ commit** — from that point, the git history of this file is the record of chan | 2026-06-19 | FR-16 | `run_id` added to the telemetry payload as the manifest-to-logs correlation key; per-call records routed to a sidecar JSONL with aggregates in the run manifest | design review; ADR-015, ADR-022 | | 2026-06-19 | FR-22 | Deferred to v1.1. v1 ships call and token ceilings with a confirmation gate in place of cost estimation and a spend ceiling | design review; ADR-027 | | 2026-06-19 | NFR-2 | Token and cost scorecard deferred to v1.1. v1 commits graph scorecards and ships prefix caching | design review | -| 2026-06-19 | NFR-5 | Full renderer guide deferred to v1.1; v1 ships a short `CONTRIBUTING.md` section. The guide waits on a second renderer to make the pattern real | design review; ADR-008 | +| 2026-06-19 | NFR-5 | Full renderer guide deferred to v1.1; v1 ships a short contributor-guide section. The guide waits on a second renderer to make the pattern real | design review; ADR-008 | | 2026-06-29 | FR-8 | Generalised — the OpenAI-compatible provider accepts any base URL, with no vendor allowlist | design review; ADR-013 | | 2026-08-05 | NFR-1 | Retired. Number retained so existing references resolve | — |