--pii-ml-language pl and ko load a spaCy model whose NER labels Presidio cannot map, so the audit reports zero ML detections with no error (CR-A11-015) #389

Wave 1 · theme D1-4: Data audit, PII/secret detection and ingestion · highest severity High · 1 finding(s) · effort small

Summary

An operator runs forgelm audit corpus_pl.jsonl --pii-ml --pii-ml-language pl after python -m spacy download pl_core_news_lg, and likewise for ko. The pre-flight passes. The model does tag persName/PS entities, but Presidio drops them. The report shows no person/organization/location categories, so the corpus looks free of unstructured PII and the audit exits 0.

Primary finding

CR-A11-015 — --pii-ml-language pl and ko load a spaCy model whose NER labels Presidio cannot map, so the audit reports zero ML detections with no error

severity High · reachability documented-opt-in · intent contract-violation · confidence confirmed · independently confirmed · reproduced (plumbing only)

What happens. _get_presidio_analyzer (forgelm/data_audit/_pii_ml.py:99-125) builds the non-English analyzer from NlpEngineProvider with only nlp_engine_name and models. It passes no ner_model_configuration, so Presidio's built-in NerModelConfiguration applies (presidio_analyzer/nlp_engine/ner_model_configuration.py:10-28). Its model_to_presidio_entity_mapping covers only PER, PERSON, LOC, LOCATION, GPE, ORG, DATE, TIME, NORP, and a few clinical labels. SpacyNlpEngine._get_updated_entities (spacy_nlp_engine.py:262-285) keeps an unmapped label under its raw name and logs a warning. SpacyRecognizer.analyze (spacy_recognizer.py:~107) then drops any entity whose label is not in supported_entities. As a result the model produces entities, but none reach the analyzer output. Hypothesis from the task, checked and mostly refuted: a SpacyRecognizer IS registered for the requested language. AnalyzerEngine builds the registry with supported_languages=[lang] and then calls registry.add_nlp_recognizer(nlp_engine) (analyzer_engine.py:~83-92). That loops over nlp_engine.get_supported_languages() and creates one SpacyRecognizer per language (recognizer_registry.py add_nlp_recognizer). So de/es/fr/it/pt/ru/xx (labels PER/LOC/ORG, plus MISC) do map to PERSON/LOCATION/ORGANIZATION and work. en, nl, zh and ja use the OntoNotes label set (PERSON/GPE/LOC/ORG) and also map. Two entries of _SPACY_MODEL_FOR_LANGUAGE fail. Label sets come from the explosion/spacy-models meta JSON at version 3.8.0 (read via raw.githubusercontent, no weights downloaded): - pl_core_news_lg: date, geogName, orgName, persName, placeName, time. None are mapped. - ko_core_news_lg: DT, LC, OG, PS, QT, TI. None are mapped. For these, detect_pii_ml returns {}. _require_presidio passes because it checks only that the language is registe…

Where.

• forgelm/data_audit/_pii_ml.py:40-53
• forgelm/data_audit/_pii_ml.py:128-170
• forgelm/data_audit/_pii_ml.py:170-179
• forgelm/data_audit/_aggregator.py:156-158
• docs/usermanuals/en/data/pii-ml.md:93
• venvs/extras/lib/python3.13/site-packages/presidio_analyzer/nlp_engine/ner_model_configuration.py:8-28
• venvs/extras/lib/python3.13/site-packages/presidio_analyzer/nlp_engine/spacy_nlp_engine.py:242-290
• venvs/extras/lib/python3.13/site-packages/presidio_analyzer/predefined_recognizers/nlp_engine_recognizers/spacy_recognizer.py:98-121
• venvs/extras/lib/python3.13/site-packages/presidio_analyzer/predefined_recognizers/nlp_engine_recognizers/spacy_recognizer.py:108-112
• forgelm/data_audit/_pii_ml.py:149-179

Recommended fix. Pass an explicit ner_model_configuration to NlpEngineProvider that extends the mapping per model. For example, add persName to PERSON, placeName and geogName to LOCATION, orgName to ORGANIZATION for pl; PS to PERSON, LC to LOCATION, OG to ORGANIZATION for ko. Alternatively, drop pl and ko from the map until they are mapped. In either case add a pre-flight that compares the loaded model's NER labels (nlp.get_pipe('ner').labels) against the mapping, and fail or warn when none map. Replace the manual's 'aborts ... instead of silently returning zero findings' sentence, and add a test with a real label set.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}