User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR) (CR-A12-004)

Wave 1 · theme D1-4: Data audit, PII/secret detection and ingestion · highest severity High · 1 finding(s) · effort trivial

Summary

A new user pastes the quick example, gets exit 1, and sees none of the advertised output.

Primary finding

CR-A12-004 — User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR)

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. docs/usermanuals/en/data/ingestion.md:26-37 (TR :26-37) shows 'forgelm ingest ./policies/ --recursive --strategy markdown --chunk-tokens 1024 --all-mask --output data/policies.jsonl' with no --tokenizer. ingest_path raises ValueError '--chunk-tokens requires --tokenizer MODEL_NAME' (ingestion.py:2393-2396) and the CLI exits 1. Dynamic check on the snapshot: rc=1, 'Ingest failed: --chunk-tokens requires --tokenizer MODEL_NAME ...'. The four tick-mark lines ('scanned 47 files', 'extracted 12,240 chunks (avg 743 tokens)', 'masked 18 PII matches', 'wrote ... (8.2 MB)') do not exist; the real text output is the 'Ingestion summary' block from summarize_result (ingestion.py:2783-2802: Output JSONL, Files, Chunks, Total chars, By format, Note, Next). The example is the first command a new user copies. tools/check_cli_help_consistency.py passes it because it validates argparse syntax only, not cross-flag requirements.

Where.

Evidence.

fpy -m forgelm ingest t/in --recursive --strategy markdown --chunk-tokens 1024 --all-mask --output t/out1.jsonl -> rc=1; summarize_result format in snapshot.

Recommended fix. Add --tokenizer to the example and replace the output block with the real summarize_result text (or the JSON envelope).

Acceptance criteria

A regression test reproduces the failure scenario and asserts the corrected behaviour.
Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}

Where to start

Thanks for taking a look. Claim it by starting a comment with /assign so nobody duplicates the work.

The first command on the Document Ingestion page fails. --chunk-tokens needs --tokenizer, so a user who pastes it gets Ingest failed: --chunk-tokens requires --tokenizer MODEL_NAME ... and exit code 1. The output shown under it (✓ scanned 47 files ...) is also something the tool never prints.

Files. The "Quick example" in docs/usermanuals/en/data/ingestion.md (lines 26-35) and the same block in docs/usermanuals/tr/data/ingestion.md ("Hızlı örnek", lines 26-35). The rule and the real output come from forgelm/ingestion.py: search for requires --tokenizer and for summarize_result.

What to change (already decided).

Replace the block in both files with the one below.
Keep --chunk-tokens 1024 and add --tokenizer "Qwen/Qwen2.5-7B-Instruct", the form docs/guides/ingestion.md uses. The rest of the page is written around --chunk-tokens, so do not switch to --chunk-size.
The output is the real summarize_result layout; the numbers are illustrative.
CLI output is English only, so the Turkish page gets the identical block and no Turkish text changes.

$ forgelm ingest ./policies/ \
    --recursive \
    --strategy markdown \
    --chunk-tokens 1024 \
    --tokenizer "Qwen/Qwen2.5-7B-Instruct" \
    --all-mask \
    --output data/policies.jsonl
Ingestion summary
  Output JSONL  : data/policies.jsonl
  Files (in/out): processed=47  skipped=0
  Chunks        : 12240
  Total chars   : 28410553
  By format     : docx=8, md=27, pdf=12
  Note          : PII masking redacted 18 span(s): email=11, phone=7
  Note          : Secrets masking enabled — no credentials detected in this corpus

Next: forgelm audit data/policies.jsonl --output ./audit/
Or train: forgelm --config <your.yaml>  (set data.dataset_name_or_path: data/policies.jsonl)

You can check the layout without downloading a model: point --tokenizer at any tokenizer directory you have locally.

Test. Add a test to tests/test_usermanual_reference_consistency.py, modelled on TestGdprPageStaysExecutable, and parametrize it over lang in ("en", "tr").

Read docs/usermanuals/<lang>/data/ingestion.md and take the first shell block that contains forgelm ingest.
Join the \ continuation lines and split the command with shlex.split.
Assert that --chunk-tokens comes with --tokenizer, and that the page shows the labels summarize_result prints (Ingestion summary, Files (in/out)).

The test fails today and passes after the fix.

pytest tests/test_usermanual_reference_consistency.py --no-cov
python3 tools/check_cli_help_consistency.py --strict
python3 tools/check_bilingual_code_blocks.py --strict
python3 tools/check_bilingual_parity.py --strict
python3 tools/check_usermanual_self_contained.py --strict

Add one line under ## [Unreleased] → ### Fixed in CHANGELOG.md.

Open the PR against development with Fixes #395 (finding CR-A12-004).

/assign

@lesbass

Assigned to @lesbass. Thank you! Please open the pull request against development and link it with Fixes #395 (or Refs #395 if it resolves only part of a grouped issue); see Working on an issue. If you cannot continue, comment /unassign so someone else can take it.

fix(docs): make the ingestion quick-start example runnable (HodeTech#395

Navigation Menu

Repository navigation

User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR) (CR-A12-004) #395

Summary

Primary finding

CR-A12-004 — User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR)

Acceptance criteria

Where to start

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR) (CR-A12-004) #395

Description

Summary

Primary finding

CR-A12-004 — User-manual quick example fails with exit 1 and prints output the tool never produces (EN and TR)

Acceptance criteria

Activity

cemililik commented on Oct 22, 2026

Where to start

lesbass commented on Oct 22, 2026

github-actions commented on Oct 22, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions