diff --git a/.claude/skills/cut-release/SKILL.md b/.claude/skills/cut-release/SKILL.md index 0828967d..bdf8dd1a 100644 --- a/.claude/skills/cut-release/SKILL.md +++ b/.claude/skills/cut-release/SKILL.md @@ -174,7 +174,8 @@ If the install fails or returns the wrong version, investigate the workflow run. ### 1. Announce -Template in [docs/marketing/05_content_strategy.md](../../../docs/marketing/05_content_strategy.md) under "New Release": +Use this template (the maintainer keeps a longer canonical version in +local working memory; the spirit is the same): ``` 📦 ForgeLM v0.4.0 released! @@ -205,7 +206,10 @@ git push ### 3. Update marketing roadmap metrics -[`docs/marketing/marketing_strategy_roadmap.md`](../../../docs/marketing/marketing_strategy_roadmap.md) — metrics row for the release month. +The maintainer keeps a marketing roadmap in local working memory +(gitignored). Add a metrics row for the release month there as part +of the release-day checklist; nothing on the public surface needs to +change. ### 4. Close the phase (if applicable) diff --git a/.claude/skills/sync-bilingual-docs/SKILL.md b/.claude/skills/sync-bilingual-docs/SKILL.md index 9dc0f611..cde9b2e5 100644 --- a/.claude/skills/sync-bilingual-docs/SKILL.md +++ b/.claude/skills/sync-bilingual-docs/SKILL.md @@ -17,12 +17,15 @@ Any change to these paths or their siblings: Do **not** use for: - `docs/standards/*` (English only) -- `docs/marketing/*` (local-only, mixed) -- `docs/analysis/*` (whichever language suits the source) +- `docs/marketing/*` (gitignored working-memory; mixed languages) - `docs/qms/*` (English only) - `docs/design/*` (English only) - `docs/guides/*` (English only for now — Turkish is future work) +Do **not** reference any file under gitignored working-memory directories +(`docs/marketing/`, `docs/analysis/`) from the bilingual docs you edit — +those paths are local-only and won't resolve in fresh clones. + ## Required reading 1. [docs/standards/localization.md](../../../docs/standards/localization.md) — the full policy diff --git a/.codacy.yaml b/.codacy.yaml new file mode 100644 index 00000000..c3e81c9e --- /dev/null +++ b/.codacy.yaml @@ -0,0 +1,24 @@ +# Codacy configuration — exclusion paths for generated artefacts. +# +# The exclusions below pin two categories that produce false-positive +# Codacy findings for this project: +# +# 1. ``site/js/wizard_defaults.js`` is a build-time artefact emitted by +# ``tools/generate_wizard_defaults.py`` from Pydantic field defaults +# flagged with ``json_schema_extra={"wizard": True}`` (F1 review-cycle 3). +# PMD's ``InaccurateNumericLiteral`` rule flags scientific notation +# like ``2e-05``; the value comes from Python's ``json.dumps`` which +# chooses scientific notation for small floats — and the JS consumer +# canonicalises with ``Number.toExponential()`` so precision is never +# operator-visible. The CI guard +# ``tools/check_wizard_defaults_sync.py`` already verifies the +# generated content matches the schema byte-for-byte; secondary +# static analysis adds no signal. +# +# 2. ``forgelm/wizard/_defaults.json`` — same generator output for the +# Python consumer. Plain JSON, no analysis applies. +# +# Codacy's analysis YAML format mirrors GitHub Actions' path globs. +exclude_paths: + - site/js/wizard_defaults.js + - forgelm/wizard/_defaults.json diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index ea32c5d3..dac71126 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -258,7 +258,25 @@ jobs: run: | # pip-audit returns non-zero on any finding; capture the JSON # and apply our own severity policy via tools/check_pip_audit.py. - pip-audit --format json --output /tmp/pip-audit.json || true + # + # Vulnerability ignores (each ignore must carry a written + # justification + remove-after condition; review at every + # release cycle). Issue #37 tracks the active set. + # + # CVE-2026-1839 — transformers, fix: 5.0.0rc3 (release candidate). + # ForgeLM's pyproject pins ``transformers>=4.38.0,<5.0.0`` and + # the 5.x branch is a major version bump that breaks downstream + # callers (TRL adapter signature changes + tokenizer-config + # API drift). The CVE has no 4.x backport at the time of + # ignore (verified 2026-05-10). Stop-gap until either (a) + # transformers ships a 4.x point release with the fix or + # (b) ForgeLM cuts a tracked major-version-bump cycle that + # raises the upper bound. Re-evaluate at each release. + pip-audit \ + --format json \ + --output /tmp/pip-audit.json \ + --ignore-vuln CVE-2026-1839 \ + || true python3 tools/check_pip_audit.py /tmp/pip-audit.json - name: bandit (static security analysis) diff --git a/.gitignore b/.gitignore index 5d837dd9..f8c2b4cd 100644 --- a/.gitignore +++ b/.gitignore @@ -1,33 +1,9 @@ -# Internal strategy, marketing, and analysis docs (not public) +# Gitignored working-memory directories — local-only PR-cycle audit +# notes, external-repo comparisons, drafts. See +# ``docs/standards/documentation.md`` ("Working-memory directories") +# for the rule: no public-doc / production-code references allowed. docs/marketing/ docs/analysis/ -# Explicit exceptions: a small set of v0.5.5-closure planning artifacts -# is exposed publicly (referenced from closure PRs and the roadmap). The -# rest of docs/analysis/** stays internal — including all subdirectories -# (QKV-Core, Trion, autoresearch, proposals, ...) and every other file -# under code_reviews/. -# -# Pattern explanation: -# 1. docs/analysis/ — ignore everything under this dir -# 2. !docs/analysis/ — re-allow the directory itself (so we can descend) -# 3. docs/analysis/* — ignore every direct entry (subdirs + files) -# 4. !docs/analysis/code_reviews/ — re-allow code_reviews/ subdir only -# 5. docs/analysis/code_reviews/* — ignore every entry in code_reviews/ -# 6. ! — re-include the 3 closure planning artifacts -!docs/analysis/ -docs/analysis/* -!docs/analysis/code_reviews/ -docs/analysis/code_reviews/* -!docs/analysis/code_reviews/closure-plan-202604300906.md -!docs/analysis/code_reviews/master-review-opus-202604300906.md -!docs/analysis/code_reviews/split-design-data_audit-cli-202604300906.md -!docs/analysis/code_reviews/wave2b-final-merge-review-prompt.md -!docs/analysis/code_reviews/wave3-review-prompt.md -!docs/analysis/code_reviews/wave3-followup-review-prompt.md -!docs/analysis/code_reviews/iso27001-soc2-alignment-202605052315.md -!docs/analysis/code_reviews/wave4-review-prompt.md -!docs/analysis/code_reviews/wave4-followup-review-prompt.md -!docs/analysis/code_reviews/wave5-review-prompt.md # Generated by tools/build_usermanuals.py — produced from # docs/usermanuals//**/*.md during the CI deploy. Not committed; diff --git a/.markdownlint.json b/.markdownlint.json index 54e5902d..b410e052 100644 --- a/.markdownlint.json +++ b/.markdownlint.json @@ -2,7 +2,9 @@ "//": "ForgeLM markdownlint policy. Consumed by Codacy's markdownlint scanner (loadbearing — without this config Codacy emits MD051 across every SPA hash-router link in docs/usermanuals/).", "//1": "MD051 (link-fragments-valid): docs use SPA hash-router routes like `#/data/audit` for in-app navigation; markdownlint's anchor-resolution doesn't understand SPA conventions and would flag every cross-reference.", "//2": "MD014 (commands-show-output): shell examples in operator docs deliberately show only the command (`$ forgelm audit ...`) without sample output — output snippets rot quickly across versions and would force a doc update on every CLI banner change.", + "//3": "MD013 (line-length): bilingual (EN+TR) reference + design docs intentionally carry long lines for paragraph-style explanations, table cells, and prose with long inline links. The 80-col limit is a code-style holdover that hurts readability of documentation; the project's bilingual_parity guard already enforces structural alignment, and the rest of the prose stays human-readable at any reasonable width.", "default": true, "MD051": false, - "MD014": false + "MD014": false, + "MD013": false } diff --git a/CHANGELOG.md b/CHANGELOG.md index 192f8688..0fca7c29 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,14 +6,269 @@ All notable changes to ForgeLM are documented here. ### Added +- **CLI wizard parity-with-web (Phase 22).** `forgelm --wizard` now + covers the same nine user-visible stages as the in-browser wizard + at `forgelm.dev/quickstart` and produces the same generated YAML. + Internal step IDs and the trainer-vs-model order may differ between + the two surfaces by design — both flows reach an equivalent + `ForgeConfig` shape; the divergence is documented inline at + `forgelm/wizard/_orchestrator.py`. The CLI flow is welcome → + use-case → model → strategy → trainer → dataset → training-params + → compliance → evaluation, with `back` / `b` to navigate backwards + and `reset` / `r` to clear in-memory state. The wizard module was + split into a sub-package (`forgelm/wizard/`) for orchestrator / + state / collectors / BYOD / IO concerns. + - **Idempotent re-run:** `forgelm --wizard --wizard-start-from + /path/to/existing.yaml` reads the YAML, validates it against + `ForgeConfig` up front, and seeds the wizard with the loaded + values so a bare Enter at every prompt keeps the operator's + earlier answer. Save flow defaults to the same path; existing + overwrite confirmation still fires. + - **State persistence:** snapshot at + `$XDG_CACHE_HOME/forgelm/wizard_state.yaml` (or + `~/.cache/forgelm/wizard_state.yaml`) saved after every completed + step; resume on next launch when the snapshot is present and + schema-compatible. Snapshot is `chmod 0o600` and cleared on + successful completion. Atomic writes via temp file + `fsync` + + `os.replace` prevent half-written state under SIGKILL / power + loss. + - **Schema-driven defaults:** wizard-relevant fields in + `forgelm/config.py` are flagged with + `json_schema_extra={"wizard": True}`. A generator script + (`tools/generate_wizard_defaults.py`) walks the schema and writes + `forgelm/wizard/_defaults.json` (consumed by the CLI wizard via + `importlib.resources`) and `site/js/wizard_defaults.js` (consumed + by the web wizard's `defaultState()`). A CI guard + (`tools/check_wizard_defaults_sync.py`) fails the run on schema- + vs-shipped-JSON drift. Hardcoded fallbacks survive only when the + JSON is missing entirely (broken pip install). + - **Trainer-specific hyperparameters:** `dpo_beta`, `simpo_beta` / + `simpo_gamma`, `kto_beta`, `orpo_beta`, `grpo_num_generations`, + `grpo_max_completion_length`, `grpo_reward_model` surface per + `trainer_type`. SFT short-circuits. + - **PEFT method breadth:** the strategy step offers all four + schema-supported `lora.method` values (`lora`, `dora`, `pissa`, + `rslora`) plus GaLore as a separate axis. All six `galore_optim` + Literal values surfaced, including the three `_layerwise` + variants. RoPE scaling adds `longrope` (full 4-of-4 schema + coverage). + - **Compliance depth:** `compliance` (Article 11 + Annex IV §1) + plus optional `risk_assessment` (Article 9), `data.governance` + (Article 10), `retention` (GDPR Article 5(1)(e) + 17), + `monitoring` (Article 12 + 17), `evaluation.benchmark`, + `evaluation.llm_judge`, `synthetic` blocks all configurable from + the wizard. + - **High-risk auto-coercion:** `risk_classification ∈ {high-risk, + unacceptable}` automatically enables `evaluation.safety` + + `evaluation.require_human_approval` with a visible operator + notice — front-stops the schema-side `F-compliance-110` + `ConfigError`. + - **Webhook URL parsing with SSRF preflight:** single prompt + accepts a literal URL or `env:VAR_NAME` reference; the URL is + `urlparse`-validated, HTTPS recommended, and loopback / RFC1918 + / link-local destinations rejected up front by reusing + `forgelm._http._is_private_destination` so typos like + `http://10.0.0.1/x` fail at config time instead of training time. + - **Configuration summary:** the wizard prints the full generated + YAML alongside the labelled headline so the operator sees every + block (webhook / evaluation / compliance / risk / retention / + monitoring) without `cat`-ing the file. + - **Step-diff preview:** each completed step prints + `+ key.path: value` / `~ key.path: before → after` so the operator + sees exactly what changed mid-flow. + - **Beginner / expert toggle:** beginner mode prefixes each step + with a 2-3-line tutorial paragraph; expert mode is silent. + - **Use-case integration:** the curated quickstart-template list is + available as Step 2 of the full flow (in addition to the existing + prelude shortcut), seeding sensible defaults for later steps + without locking anything down. Use-case keys mirror + `forgelm/quickstart.py::TEMPLATES` exactly; `quickstart.py` is + the single source of truth. + - **Pre-flight checklist:** GPU / VRAM / dataset / risk-tier + signals surface before the configuration summary, calling out + common operator errors (low-VRAM full-precision, missing local + file, strict tier without safety eval). + - **Distinct exit code for wizard cancel:** new + `EXIT_WIZARD_CANCELLED = 5`. Clean cancels exit `5` instead of + `0` so CI can distinguish "wizard finished" from "wizard never + produced output". Public exit-code surface is now `0–5`. + - **Best-effort readline:** arrow-key line editing + history on + Linux/macOS via stdlib `readline` import; Windows unaffected. + - **Validate-on-exit:** the wizard runs `ForgeConfig.model_validate` + on the saved YAML before declaring success. Schema rejections + surface inline with the offending field rather than 30 minutes + into a failed training run. + - **Overwrite confirmation + auto-suffix:** the save flow detects + pre-existing files, asks before clobbering, and falls back to the + next free `_2.yaml` / `_3.yaml` slot when declined. + - **Non-tty stdin refusal:** `forgelm --wizard < answers.txt` used + to silently produce empty configs; the wizard now refuses to + launch on a non-tty stdin and points the operator at + `forgelm quickstart