Skip to content

Code does not match docs/usermanuals/en/operations/air-gap.md: --offline: W&B/MLflow/Comet Disabled; OpenAI/Anthropic judges Disabled; Webhooks Disabled (or restricted via webhook… (CR-MISC-004) #100

Open

Description

@cemililik

Wave 0 · theme D0-5: Documented controls that are not implemented · highest severity High · 1 finding(s) · effort not estimated

Summary

The document states: --offline: W&B/MLflow/Comet Disabled; OpenAI/Anthropic judges Disabled; Webhooks Disabled (or restricted via webhook.allow_private). Observed in the code: apply_offline_flag only sets the HF*_OFFLINE env vars (forgelm/cli/_config_load.py:50-57). judge.py and webhook.py contain no 'offline' check, and WANDB_MODE is never set. The field is webhook.allow_private_destinations (config.py:1488).

Primary finding

CR-MISC-004 — Code does not match docs/usermanuals/en/operations/air-gap.md: --offline: W&B/MLflow/Comet Disabled; OpenAI/Anthropic judges Disabled; Webhooks Disabled (or restricted via webhook.allow_private)

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · already tracked: KNOWN-OPEN CLI-04 (inferred) — docs/roadmap/phase-16-trust-surface-hardening.md:483-489,509-510

What happens. air-gapped operators rely on --offline to stop judge, webhook and tracker traffic, but it only sets the HF_*_OFFLINE variables; whether the code or the page is wrong is for the refuters

Where.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.
  • The release note names the contract this fix restores.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    backlog: knownAlready tracked in docs/roadmap
    severity: highWrong behaviour or a false claim users rely on
    source: review-2026-10Found by the October 2026 code and documentation review
    wave: 0Roadmap wave 0: immediate (patch release)
    on Oct 9, 2026
  2. cemililik commented on Oct 17, 2026

    @cemililik
    CollaboratorAuthor

    Progress on development — verified 2026-10-18 at 6c55900.

    Fixed (documentation only):

    • the air-gap manual page (EN/TR) no longer claims the flag disables webhooks, the API judge or the API teacher, and uses the real field name allow_private_destinations (d138664, 88d04f9, b03fea1)

    Still open:

    • the --offline help text in forgelm/cli/_parser.py and docs/usermanuals/en/reference/cli.md still say "disable all HF Hub network calls"
    • _apply_offline_flag still runs after the subcommand is dispatched (planned as CLI-04 in Phase 16 step S7)
    • the "Comet" row and a regression test

    Everything else in this issue is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: dev-toolingCode area: dev-toolingbacklog: knownAlready tracked in docs/roadmapbugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 0Roadmap wave 0: immediate (patch release)

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions