feat: Phase 10 post-training completion (v0.4.0)

.github/workflows/nightly.yml

@@ -5,13 +5,14 @@ on:
     - cron: "0 3 * * *"  # 03:00 UTC daily
   workflow_dispatch:  # Manual trigger
 
-permissions:
-  issues: write  # Required for creating failure notification issues
+# No workflow-level permissions — each job declares the minimal set it needs.
 
 jobs:
   test-latest-deps:
     name: Test with latest dependencies
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     strategy:
       fail-fast: false
       matrix:
@@ -56,6 +57,8 @@ jobs:
   test-min-deps:
     name: Test with minimum supported versions
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     steps:
       - uses: actions/checkout@v5
 
@@ -87,6 +90,9 @@ jobs:
     needs: [test-latest-deps, test-min-deps]
     if: failure()
     runs-on: ubuntu-latest
+    permissions:
+      contents: read  # github-script needs to read the repo
+      issues: write   # required for creating failure notification issues
     steps:
       - name: Create issue on failure
         uses: actions/github-script@v8

.github/workflows/publish.yml

@@ -4,12 +4,13 @@ on:
   release:
     types: [published]
 
-permissions:
-  id-token: write  # Required for trusted publishing
+# No workflow-level permissions — each job declares the minimal set it needs.
 
 jobs:
   build:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read  # checkout needs read access to the release ref
     steps:
       - uses: actions/checkout@v5
 
@@ -42,6 +43,9 @@ jobs:
     environment:
       name: testpypi
       url: https://test.pypi.org/p/forgelm
+    permissions:
+      contents: read  # download-artifact reads workflow run state
+      id-token: write  # required for trusted publishing to TestPyPI
     steps:
       - uses: actions/download-artifact@v5
         with:
@@ -61,6 +65,9 @@ jobs:
     environment:
       name: pypi
       url: https://pypi.org/p/forgelm
+    permissions:
+      contents: read  # download-artifact reads workflow run state
+      id-token: write  # required for trusted publishing to PyPI
     steps:
       - uses: actions/download-artifact@v5
         with:

CHANGELOG.md

@@ -4,6 +4,103 @@ All notable changes to ForgeLM are documented here.
 
 ## [Unreleased]
 
+## [0.4.0] — 2026-04-25
+
+### Added
+
+**Post-Training Completion (Phase 10)**
+
+- **`forgelm/inference.py`** — Shared generation primitives for all post-training features:
+  - `load_model(path, adapter, backend, load_in_4bit, load_in_8bit, trust_remote_code)` — loads HF model + tokenizer; optional PEFT adapter merge via `merge_and_unload()`; unsloth backend support
+  - `generate(model, tokenizer, prompt, *, messages, system_prompt, history, max_new_tokens, temperature, top_k, top_p, repetition_penalty)` — non-streaming text generation
+  - `generate_stream(...)` — streaming via `TextIteratorStreamer` in daemon thread; yields token chunks
+  - `logit_stats(logits)` — returns `{entropy, top1_prob, effective_vocab}` for token-level confidence inspection
+  - `adaptive_sample(logits, temperature, top_k, top_p, entropy_threshold)` — greedy below entropy threshold, nucleus sampling above
+  - `_build_prompt` — uses `tokenizer.apply_chat_template` when available; falls back to `"role: content\n"` join
+
+- **`forgelm/chat.py`** — Interactive terminal REPL (`ChatSession` class + `run_chat()` entry point):
+  - Streaming output by default; `--no-stream` flag for non-streaming
+  - Slash commands: `/reset`, `/save [file]`, `/temperature N`, `/system [prompt]`, `/help`, `/exit`
+  - History management with 50-turn cap (`_MAX_HISTORY_PAIRS`)
+  - Optional `rich` rendering via `pip install forgelm[chat]`
+  - Optional `--safety` flag routes each response through Llama Guard
+
+- **`forgelm/fit_check.py`** — VRAM pre-flight advisor:
+  - `estimate_vram(config)` → `FitCheckResult(verdict, estimated_gb, available_gb, breakdown, recommendations)`
+  - Verdicts: `FITS` (< 85% GPU), `TIGHT` (85-95%), `OOM` (> 95%), `UNKNOWN` (no GPU)
+  - Architecture loaded via `transformers.AutoConfig`; fallback size-hint dict for 7b/8b/13b/70b families
+  - VRAM components: base weights + LoRA adapter + optimizer state (AdamW/8-bit/GaLore-aware) + activations (gradient-checkpointing divides by √layers)
+  - `format_fit_check(result)` — human-readable summary; `--output-format json` for CI/CD
+  - Hypothetical mode when no CUDA detected — still estimates based on architecture
+
+- **`forgelm/export.py`** — GGUF model export:
+  - `export_model(model_path, output_path, *, format, quant, adapter, update_integrity, extra_args)` → `ExportResult`
+  - Wraps `llama-cpp-python`'s `convert_hf_to_gguf.py` — no reimplementation of conversion logic
+  - Supported quantizations: `q2_k`, `q3_k_m`, `q4_k_m`, `q5_k_m`, `q8_0`, `f16`
+  - **K-quant note**: `q2_k`/`q3_k_m`/`q4_k_m`/`q5_k_m` require a two-step flow.
+    `forgelm export ... --quant q4_k_m model.gguf` produces an intermediate
+    `model.f16.gguf`; run `llama-quantize model.f16.gguf model.gguf Q4_K_M`
+    afterward to obtain the K-quant. The `ExportResult.quant` field reflects
+    what was actually written (so `model_integrity.json` SHA-256 stays honest)
+  - Adapter merge: loads base + PEFT, saves merged fp16 weights before conversion
+  - `_sha256_file` — chunked 64 KB reads for large models
+  - `_update_integrity_manifest` — appends export artifact (path, quant, sha256, size_bytes) to `model_integrity.json`
+  - Optional dependency: `pip install forgelm[export]` (`llama-cpp-python>=0.2.90`)
+
+- **`forgelm/deploy.py`** — Deployment config file generation:
+  - `generate_deploy_config(model_path, target, output_path, *, system_prompt, max_length, temperature, top_k, top_p, ...)` → `DeployResult`
+  - Target `ollama`: Modelfile with FROM, SYSTEM (double-quote escaped), PARAMETER directives
+  - Target `vllm`: YAML engine config with GPU memory utilization, dtype, trust_remote_code
+  - Target `tgi`: docker-compose.yaml with GPU resource reservation, port mapping, max-input/total-length
+  - Target `hf-endpoints`: JSON spec with model repository, task, compute instance, region, framework
+  - Case-insensitive target matching; default output filenames per target
+
+- **CLI subcommands** (`forgelm/cli.py`):
+  - `forgelm chat MODEL_PATH [--adapter] [--system] [--temperature] [--max-new-tokens] [--safety] [--no-stream] [--load-in-4bit] [--load-in-8bit] [--trust-remote-code] [--backend]`
+  - `forgelm export MODEL_PATH --output FILE [--format gguf] [--quant q4_k_m] [--adapter] [--no-integrity-update]`
+  - `forgelm deploy MODEL_PATH --target TARGET [--output FILE] [--system] [--max-length] [--temperature] [--top-k] [--top-p] [--trust-remote-code]`
+  - `forgelm --config CONFIG --fit-check [--output-format json]`
+  - All subcommands work without `--config`; backward-compatible with existing flat CLI
+
+- **Optional extras** in `pyproject.toml`:
+  - `forgelm[export]` — `llama-cpp-python>=0.2.90` (non-Windows)
+  - `forgelm[chat]` — `rich>=13.0.0`
+
+- **New test modules**:
+  - `tests/test_inference.py` — 16 tests covering `_build_prompt`, `_to_messages`, `logit_stats`, `adaptive_sample`, `load_model`, `generate` with custom torch stub (no GPU required)
+  - `tests/test_fit_check.py` — 18 tests covering parameter estimation, VRAM components, GPU scenarios (no CUDA, 4 GB, 80 GB), `format_fit_check`
+  - `tests/test_export.py` — 12 tests covering SHA-256, integrity manifest, GGUF export flow with subprocess mock
+  - `tests/test_deploy.py` — 21 tests covering all 4 target generators and `generate_deploy_config` integration
+  - `tests/test_cli_phase10.py` — 22 tests covering `--fit-check`, all deploy targets, export subcommand, chat subcommand, subcommand routing
+
+### Changed
+
+- **`forgelm/__init__.py`** — version bumped to `0.4.0`
+- **`forgelm/cli.py`** — added subparser architecture with `chat`, `export`, `deploy` subcommands; added `--fit-check` flag; `KeyboardInterrupt` caught in chat dispatch for graceful exit
+- **`forgelm/wizard.py`** — (no changes needed; Phase 10 features are all CLI-driven, not wizard-driven)
+
+### Breaking
+
+- **`forgelm.compliance.export_compliance_artifacts`** signature changed from
+  `(manifest, config, output_dir)` to `(manifest, output_dir)`. The `config`
+  argument was unused (the manifest already contains all derived values).
+  External callers must drop the second positional argument.
+- **`forgelm.export.export_model`** keyword `format=` renamed to
+  `output_format=` to avoid shadowing the `format` builtin. Update
+  `export_model(..., format="gguf", ...)` → `export_model(...,
+  output_format="gguf", ...)`.
+- **`forgelm.deploy.generate_deploy_config`** parameter list collapsed from
+  18 → 11 args. The HF Endpoints fields (task/instance_size/instance_type/
+  region/framework/vendor) are now grouped as
+  `hf_endpoints: HFEndpointsOptions = None`; sampling defaults
+  (temperature/top_k/top_p) are grouped as
+  `sampling: SamplingOptions = None`. Pass instances of those dataclasses
+  instead of the individual kwargs.
+
+---
+
+## [0.3.1rc1] — 2026-03-28 (included in v0.4.0 branch)
+
 ### Added
 - **Engineering standards** (`docs/standards/`) — 9 standard documents: coding, architecture, error-handling, logging-observability, testing, documentation, localization, code-review, release.
 - **AI agent skills** (`.claude/skills/`) — 6 task-specific SKILL.md checklists: add-config-field, add-trainer-feature, add-test, sync-bilingual-docs, cut-release, review-pr.
@@ -14,6 +111,18 @@ All notable changes to ForgeLM are documented here.
 - **docs/ reorganization** — Reference docs moved to `docs/reference/`, design specs to `docs/design/`. All internal links updated (29 link fixes).
 - **Roadmap refactored** — `docs/roadmap.md` reduced from 910 to 78 lines; phase details moved to `docs/roadmap/` subdirectory.
 
+### Fixed (Security & Config Hardening)
+- Webhook URLs excluded from HuggingFace Hub model cards — prevents credential leaks
+- User-supplied strings sanitized before Markdown template embedding (content injection prevention)
+- All 19 Pydantic sub-models enforce `extra="forbid"` — YAML typos are errors, not silent bugs
+- Deprecated `lora.use_dora` / `lora.use_rslora` booleans auto-normalize to `lora.method` with warnings
+- Audit log hash chain restores continuity across process restarts
+- Compliance manifests correctly report pre-OOM-recovery batch size
+- GRPO reward model path correctly wrapped as callable
+- Safety classifier receives full `[INST] prompt [/INST] response` context
+- Extension-less files raise clear `ValueError` instead of silently loading wrong format
+- TIES tie-breaking fixed; DARE now deterministic with `seed=42`
+
 ## [0.3.0] — 2026-03-28
 
 ### Added

README.md

@@ -26,6 +26,12 @@
 - **LLM-as-Judge**: API-based (OpenAI) or local model scoring for quality assessment
 - **Auto-Revert**: Automatically discard models that fail loss, benchmark, or safety thresholds
 
+### Post-Training (v0.4.0)
+- **Interactive Chat**: `forgelm chat ./model` — streaming REPL with `/reset`, `/save`, `/temperature`, `/system` commands; optional Llama Guard safety routing
+- **GGUF Export**: `forgelm export ./model --quant q4_k_m` — wraps `llama-cpp-python` converter; 6 quant levels; SHA-256 appended to integrity manifest
+- **Deployment Configs**: `forgelm deploy ./model --target ollama|vllm|tgi|hf-endpoints` — generates ready-to-use config files; does not start the server
+- **VRAM Fit Check**: `forgelm --config my.yaml --fit-check` — pre-flight memory estimator; `FITS / TIGHT / OOM / UNKNOWN` verdict with recommendations
+
 ### Enterprise & MLOps
 - **Config-Driven**: Declarative YAML — built for CI/CD pipelines, not notebooks
 - **EU AI Act Compliance**: Auto-generated audit trails, data provenance (SHA-256), training manifests
@@ -44,18 +50,24 @@
 ```bash
 # Install
 pip install -e .
+pip install -e ".[export]"   # GGUF export (optional, non-Windows)
 
 # Generate config interactively
 forgelm --wizard
 
-# Or copy template and edit
-cp config_template.yaml my_config.yaml
-
 # Validate without training
 forgelm --config my_config.yaml --dry-run
 
+# Check VRAM before a long run
+forgelm --config my_config.yaml --fit-check
+
 # Train
 forgelm --config my_config.yaml
+
+# After training: chat, export, deploy
+forgelm chat ./checkpoints/final_model
+forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m
+forgelm deploy ./checkpoints/final_model --target ollama --output ./Modelfile
 ```
 
 See the [Quick Start Guide](docs/guides/quickstart.md) for a complete walkthrough.
@@ -162,10 +174,15 @@ forgelm --version                            # Show version
 ```
 forgelm/
 ├── cli.py           # CLI with 10+ modes (train, dry-run, merge, benchmark, wizard...)
-├── config.py        # Pydantic config (15 models: training, evaluation, distributed...)
+├── config.py        # Pydantic config (19 models: training, evaluation, distributed...)
 ├── data.py          # Dataset loading (SFT, DPO, KTO, GRPO formats + multi-dataset)
 ├── model.py         # Model loading (transformers, unsloth, MoE, PEFT)
 ├── trainer.py       # Training orchestration (6 trainer types via TRL, GaLore, long-context)
+├── inference.py     # Shared inference primitives (load, generate, stream, adaptive sampling)
+├── chat.py          # Interactive terminal REPL with streaming and slash commands
+├── export.py        # GGUF export via llama-cpp-python
+├── fit_check.py     # Pre-flight VRAM estimator (FITS / TIGHT / OOM / UNKNOWN)
+├── deploy.py        # Deployment config generator (Ollama, vLLM, TGI, HF Endpoints)
 ├── results.py       # TrainResult dataclass
 ├── benchmark.py     # lm-evaluation-harness integration
 ├── safety.py        # Post-training safety evaluation (Llama Guard)
@@ -180,7 +197,7 @@ forgelm/
 
 configs/deepspeed/   # ZeRO-2, ZeRO-3, ZeRO-3+Offload presets
 notebooks/           # Colab-ready Jupyter notebooks
-tests/               # 145+ unit tests across 11 test files
+tests/               # 430 passed (+34 skipped) across 30 test files
 docs/guides/         # Quickstart, alignment, CI/CD, enterprise, safety guides
 ```
 

docs/guides/quickstart.md

@@ -90,12 +90,14 @@ Before starting a long run, estimate if your config fits in GPU memory:
 
 ```bash
 forgelm --config my_config.yaml --fit-check
-# → Coming in v0.4.0: GPU: RTX 3060 12GB; Estimated peak: 10.8 GB; Verdict: ✅ FITS
+# GPU: RTX 3060 12GB — Estimated peak: 10.8 GB — Verdict: FITS
+# Or: Verdict: TIGHT — Enable gradient checkpointing and reduce batch size
+# Or: Verdict: UNKNOWN — No GPU detected (hypothetical estimate)
 ```
 
-> **Note:** `--fit-check` is planned for v0.4.0.
+Output includes a breakdown (base weights, LoRA adapter, optimizer state, activations) and ordered recommendations when memory is tight. Use `--output-format json` for CI/CD integration.
 
-For now, reduce `max_length` and `per_device_train_batch_size` if you hit OOM. The [Troubleshooting guide](troubleshooting.md) has detailed OOM solutions.
+If you hit OOM during training, the [Troubleshooting guide](troubleshooting.md) has detailed solutions.
 
 ## 6. Use Your Model
 
@@ -112,19 +114,21 @@ output = model.generate(**inputs, max_new_tokens=200)
 print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
 
-### Using Your Model (After v0.4.0)
+### Using Your Model (v0.4.0+)
 
-Starting with v0.4.0, you can interact with your trained model directly:
+Interact with and deploy your trained model directly:
 
 ```bash
-# Chat with your fine-tuned model
+# Chat with your fine-tuned model (streaming by default)
 forgelm chat ./checkpoints/final_model
 
-# Export to GGUF (for Ollama, LM Studio)
-forgelm export ./checkpoints/final_model --format gguf --quant q4_k_m --output model.gguf
+# Export to GGUF (for Ollama, LM Studio, llama.cpp)
+# Requires: pip install forgelm[export]
+forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m
 
-# Generate deployment configs
+# Generate deployment configs (no server is started)
 forgelm deploy ./checkpoints/final_model --target ollama --output ./Modelfile
+forgelm deploy ./checkpoints/final_model --target vllm --output ./vllm_config.yaml
 ```
 
 ---

docs/reference/usage.md

@@ -22,6 +22,8 @@ pip install -e ".[eval]"         # lm-evaluation-harness
 pip install -e ".[tracking]"     # W&B experiment tracking
 pip install -e ".[distributed]"  # DeepSpeed multi-GPU
 pip install -e ".[merging]"      # mergekit model merging
+pip install -e ".[export]"       # GGUF export (llama-cpp-python, non-Windows)
+pip install -e ".[chat]"         # Rich rendering in forgelm chat
 ```
 
 ## Authentication
@@ -77,6 +79,47 @@ forgelm --config my_config.yaml --resume ./checkpoints/checkpoint-500
 forgelm --config my_config.yaml --offline
 ```
 
+### VRAM Fit Check
+
+Before training, estimate whether your config fits in GPU memory:
+
+```bash
+# Text output (human-readable verdict)
+forgelm --config my_config.yaml --fit-check
+
+# JSON output (for CI/CD pipelines)
+forgelm --config my_config.yaml --fit-check --output-format json
+```
+
+Output shows: estimated peak VRAM, available VRAM (if GPU detected), verdict (`FITS` / `TIGHT` / `OOM` / `UNKNOWN`), and ordered recommendations. Falls back to hypothetical mode when no GPU is detected.
+
+### Post-Training: Chat, Export, Deploy
+
+After training, interact with and deploy your fine-tuned model directly from the CLI. These subcommands work without `--config`.
+
+```bash
+# Interactive chat REPL (streaming by default)
+forgelm chat ./checkpoints/final_model
+forgelm chat ./checkpoints/final_model --adapter ./adapter --safety
+forgelm chat ./checkpoints/final_model --system "You are a helpful assistant." --temperature 0.8
+
+# Export to GGUF (for Ollama, LM Studio, llama.cpp)
+# Requires: pip install forgelm[export]
+forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m
+forgelm export ./checkpoints/final_model --output model.gguf --quant q8_0 --adapter ./adapter
+
+# Generate deployment config files
+forgelm deploy ./checkpoints/final_model --target ollama --output ./Modelfile
+forgelm deploy ./checkpoints/final_model --target vllm --output ./vllm_config.yaml
+forgelm deploy ./checkpoints/final_model --target tgi --output ./docker-compose.yaml
+forgelm deploy ./checkpoints/final_model --target hf-endpoints --output ./endpoint.json
+forgelm deploy ./checkpoints/final_model --target ollama --output ./Modelfile --system "Be concise."
+```
+
+**Chat slash commands:** `/reset`, `/save [file]`, `/temperature N`, `/system [prompt]`, `/help`, `/exit`
+
+**Export quantization levels:** `q2_k`, `q3_k_m`, `q4_k_m` (recommended), `q5_k_m`, `q8_0`, `f16`
+
 ### Synthetic Data Generation
 
 ```bash