Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,35 @@ All notable changes to ForgeLM are documented here.
> Per-PR CHANGELOG entries below collapse into the v0.5.5 release
> notes at tag time.

### Added — Wave 2a / Phase 37 — `forgelm approvals` listing subcommand

- **`forgelm approvals --pending [--output-dir DIR]`** lists every run whose
audit log carries a `human_approval.required` event without a matching
terminal decision. Tabular text output or a JSON envelope
(`{"success": true, "pending": [...], "count": N}`) when
`--output-format json` is set, so CI can filter the queue programmatically.
- **`forgelm approvals --show RUN_ID --output-dir DIR`** prints the full
approval-gate audit chain (request → terminal decision) plus the on-disk
staging directory layout. Useful for forensic review of granted /
rejected runs and confirming the staging contents match what the
operator approved.
- Closes the Phase 9 follow-up gap from `ghost-features-analysis-20260502`
(GH-007 — listing was missing; an operator could not discover pending
runs without `ls`-ing the filesystem by hand).
- New module: `forgelm/cli/subcommands/_approvals.py`. Reuses the audit-
log parsing pattern from `_approve.py` (skip malformed lines, emit one
summary warning at end).
- 15 new tests in `tests/test_approvals_listing.py` covering: empty
audit log, three-runs-one-decided, rejected runs excluded from
pending, JSON envelope structure, table rendering, `--show` pending /
granted / unknown run_id, `--show` with no audit log, monkeypatch
facade resolution, malformed-line robustness, dispatcher defensive
double-check.
- Operator docs updated: `docs/usermanuals/{en,tr}/compliance/human-
oversight.md` "Inspecting pending runs" section now matches the
shipped output (table format, `--output-dir` flag) instead of the
pre-implementation sketch.

### Added — Wave 1 closure (Faz 9, 11, 12, 13, 25, 31, 32 — see PR description)

- **Article 14 staging directory + `forgelm approve` / `forgelm reject` (Faz 9)** —
Expand Down
31 changes: 25 additions & 6 deletions docs/usermanuals/en/compliance/human-oversight.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,18 +127,37 @@ Default is 48 hours. Set to 0 for "no timeout — wait forever" (not recommended

## Inspecting pending runs

`forgelm approvals` is the discovery counterpart to `approve` / `reject`. It scans the audit log under `--output-dir` and reports every run whose `human_approval.required` event has no matching terminal decision.

```shell
$ forgelm approvals --pending
RUN_ID REQUESTED_AT ARTIFACTS EXPIRES
abc123 2026-04-29T14:33Z checkpoints/run/artifacts/ in 47h
def456 2026-04-29T09:12Z checkpoints/sft-only/artifacts/ in 42h
$ forgelm approvals --pending --output-dir checkpoints/
Pending approvals (2):

RUN_ID AGE REQUESTED_AT STAGING
---------------- ---- ------------------------- -------
fg-abc123def456 3h 2026-04-30T11:33:10+00:00 present
fg-def456abc789 1d 2026-04-29T14:12:55+00:00 present
```

`--output-format json` returns a structured envelope (`{"success": true, "pending": [...], "count": 2}`) so CI can filter the queue programmatically.

```shell
$ forgelm approvals --show abc123
... full artifact summary including audit, benchmarks, safety, model card ...
$ forgelm approvals --show fg-abc123def456 --output-dir checkpoints/
Run: fg-abc123def456
Status: pending

Audit chain (oldest first):
[2026-04-30T11:33:10+00:00] human_approval.required — require_human_approval=true

Staging contents (4 entries):
- adapter_config.json
- adapter_model.safetensors
- tokenizer.json
- tokenizer_config.json
```

A `--show` against a granted / rejected run prints the full timeline (request → decision) plus the final approver and comment. `--show` against an unknown `run_id` exits 1 with a clear error.

## Common pitfalls

:::warn
Expand Down
31 changes: 25 additions & 6 deletions docs/usermanuals/tr/compliance/human-oversight.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,18 +127,37 @@ Varsayılan 48 saat. "Timeout yok — sonsuza kadar bekle" için 0 (CI'da öneri

## Bekleyen koşuları inceleme

`forgelm approvals`, `approve` / `reject` komutlarının tamamlayıcısıdır: `--output-dir` altındaki audit log'u tarar ve `human_approval.required` event'i için terminal kararı (granted/rejected) bulunmayan tüm koşuları raporlar.

```shell
$ forgelm approvals --pending
RUN_ID REQUESTED_AT ARTIFACTS EXPIRES
abc123 2026-04-29T14:33Z checkpoints/run/artifacts/ 47s sonra
def456 2026-04-29T09:12Z checkpoints/sft-only/artifacts/ 42s sonra
$ forgelm approvals --pending --output-dir checkpoints/
Pending approvals (2):

RUN_ID AGE REQUESTED_AT STAGING
---------------- ---- ------------------------- -------
fg-abc123def456 3h 2026-04-30T11:33:10+00:00 present
fg-def456abc789 1d 2026-04-29T14:12:55+00:00 present
```

`--output-format json` yapısal bir zarf döner (`{"success": true, "pending": [...], "count": 2}`); CI bu çıktıyla kuyruğu programatik olarak filtreleyebilir.

```shell
$ forgelm approvals --show abc123
... audit, benchmark'lar, güvenlik, model card dahil tam artifact özeti ...
$ forgelm approvals --show fg-abc123def456 --output-dir checkpoints/
Run: fg-abc123def456
Status: pending

Audit chain (oldest first):
[2026-04-30T11:33:10+00:00] human_approval.required — require_human_approval=true

Staging contents (4 entries):
- adapter_config.json
- adapter_model.safetensors
- tokenizer.json
- tokenizer_config.json
```

Granted / rejected bir koşu üzerinde `--show` çalıştırıldığında tam zaman çizelgesi (talep → karar) ve son onaylayan + yorum yazdırılır. Bilinmeyen bir `run_id` üzerinde `--show` net bir hata mesajıyla 1 koduyla çıkar.

## Sık hatalar

:::warn
Expand Down
9 changes: 9 additions & 0 deletions forgelm/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@

# Parser (registrars + parse_args).
from ._parser import (
_add_approvals_subcommand, # noqa: F401 — re-export for tests
_add_approve_subcommand, # noqa: F401 — re-export for tests
_add_audit_subcommand, # noqa: F401 — re-export for tests
_add_chat_subcommand, # noqa: F401 — re-export for tests
Expand Down Expand Up @@ -113,6 +114,13 @@
# Wizard mode.
from ._wizard import _maybe_run_wizard # noqa: F401 — re-export for tests

# Approvals listing subcommand (Phase 9 follow-up).
from .subcommands._approvals import (

Check warning on line 118 in forgelm/cli/__init__.py

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

forgelm/cli/__init__.py#L118

'.subcommands._approvals._collect_pending_runs' imported but unused (F401)
_collect_pending_runs, # noqa: F401 — re-export for tests
_collect_run_audit_chain, # noqa: F401 — re-export for tests
_run_approvals_cmd, # noqa: F401 — re-export for tests
)

# Approve / reject subcommands (Article 14 human-approval gate).
from .subcommands._approve import (
_atomic_rename_or_move, # noqa: F401 — re-export for tests
Expand All @@ -123,6 +131,7 @@
_resolve_approver_identity, # noqa: F401 — re-export for tests
_run_approve_cmd, # noqa: F401 — re-export for tests
_run_reject_cmd, # noqa: F401 — re-export for tests
_staging_path_inside_output_dir, # noqa: F401 — re-export for tests + _approvals reuse
)

# Audit subcommand (+ legacy --data-audit worker).
Expand Down
3 changes: 3 additions & 0 deletions forgelm/cli/_dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ def _dispatch_subcommand(command: str, args) -> None:
elif command == "reject":
_cli_facade._run_reject_cmd(args, getattr(args, "output_format", "text"))
sys.exit(EXIT_SUCCESS)
elif command == "approvals":
_cli_facade._run_approvals_cmd(args, getattr(args, "output_format", "text"))
sys.exit(EXIT_SUCCESS)
else:
logger.error("Unrecognized subcommand: %r. This is a bug — please report it.", command)
sys.exit(EXIT_TRAINING_ERROR)
Expand Down
44 changes: 44 additions & 0 deletions forgelm/cli/_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -487,7 +487,7 @@
type=str,
required=True,
metavar="DIR",
help="Training output directory containing audit_log.jsonl and final_model.staging/.",

Check failure on line 490 in forgelm/cli/_parser.py

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal "Training output directory containing audit_log.jsonl and final_model.staging/." 3 times.

See more on https://sonarcloud.io/project/issues?id=cemililik_ForgeLM&issues=AZ3pGtP_DB_MdPLcv4N4&open=AZ3pGtP_DB_MdPLcv4N4&pullRequest=25
)
p.add_argument(
"--comment",
Expand All @@ -499,6 +499,49 @@
_add_common_subparser_flags(p, include_output_format=True)


def _add_approvals_subcommand(subparsers) -> None:
"""Article 14 follow-up: list / inspect pending approval requests.

Exactly one of ``--pending`` and ``--show RUN_ID`` must be set; argparse
enforces this with a mutually-exclusive group so the dispatcher only
sees a validated args namespace. ``--output-dir`` is required in both
modes — there is no useful default; an operator running on a freshly-
cloned workstation must point at the training output explicitly.
"""
p = subparsers.add_parser(
"approvals",
help="List pending Article 14 approval requests or inspect a single run.",
description=(
"Discovery counterpart to `forgelm approve` / `forgelm reject`. "
"`--pending` lists every run whose audit log carries a "
"`human_approval.required` event without a matching terminal "
"decision. `--show RUN_ID` prints the full approval-gate audit "
"chain plus the on-disk staging directory layout for one run."
),
)
mode = p.add_mutually_exclusive_group(required=True)
mode.add_argument(
"--pending",
action="store_true",
help="List every run awaiting an approval decision.",
)
mode.add_argument(
"--show",
type=str,
default=None,
metavar="RUN_ID",
help="Show the full audit chain + staging contents for a single run.",
)
p.add_argument(
"--output-dir",
type=str,
required=True,
metavar="DIR",
help="Training output directory containing audit_log.jsonl and final_model.staging/.",
)
_add_common_subparser_flags(p, include_output_format=True)


def _add_reject_subcommand(subparsers) -> None:
"""Article 14: discard a staged model (preserves staging dir for forensics)."""
p = subparsers.add_parser(
Expand Down Expand Up @@ -563,6 +606,7 @@
_add_verify_audit_subcommand(subparsers)
_add_approve_subcommand(subparsers)
_add_reject_subcommand(subparsers)
_add_approvals_subcommand(subparsers)

# --- Top-level flags (training / config-driven mode) ---
parser.add_argument("--config", type=str, help="Path to the YAML configuration file.")
Expand Down
Loading