Repository navigation
feat(prompt): extraction focus and content requirements settings - #895
green-dalii wants to merge 4 commits into
Conversation
Split out of #755, which bundled this with a machine-name guard that now lives in its own PR. The code is @weqoocu's; the split, and the removal described below, are mine. Two free-text fields that shape what the model looks for and writes about, without touching any contract the plugin owns: - `extractionFocus` is appended to the extraction prompt's Extraction Scope block as a domain-priority hint. It narrows what the model looks for; it does not replace the JSON protocol, the classification rules or the tag vocabulary. - `contentRequirements` is appended to entity/concept page generation through a `{{content_requirements}}` placeholder after the merge-strategy block. It controls emphasis and structure; citation, wikilink, merge and anti-hallucination rules stay binding. Empty is the default and empty adds nothing, so every vault that never configures these gets byte-identical prompts and keeps its prefix cache. ## What was removed from the original commit, and why The submitted change also carried a per-topic override layer — `settings.wikiTopics`, `activeTopicId`, a `WikiTopic` interface, and a `promptFocus.ts` resolution path that preferred a topic's own fields over the global ones. It could not run: - nothing in the settings UI creates or edits a `wikiTopics` entry, and nothing elsewhere writes the field, so it was always `[]`; - `activeTopicId` had no reader and no writer; - the only trace of the topic scope in the UI was a description string naming a scope the user could not reach. That is unreachable code by construction, and the repository's dead-code rule is to wire it or delete it before a release ships it. It is deleted here and recorded as an issue, so bringing it back means bringing the picker with it. ## i18n Eight new keys in all eleven locales, with the key sets identical so the type barrel stays honest. Japanese, Korean, German, French, Spanish, Portuguese, Italian and Russian currently carry the English text; Chinese and Traditional Chinese are translated. That gap is real and is called out rather than hidden. ## Credit The commit author line is @weqoocu's. If this merges by squash the merge commit's author becomes the PR author — me — which would understate their work; that choice belongs to the maintainer at merge time. Gate 1: lint 0, tsc 0, build clean, tests pass, css-lint 0.
…ng locales The feature shipped with the English text in eight of the eleven locales. Chinese and Traditional Chinese were translated; Japanese, Korean, German, French, Spanish, Portuguese, Italian and Russian carried the English source verbatim, so a user in any of those languages read an English settings panel describing a feature whose every neighbour was in their own language. 56 strings: 7 keys x 8 locales. The wording follows each locale's existing terms rather than a fresh glossary — `抽出` / `추출` / `Extraktion` / `Extraction` / `Extracción` / `Extração` / `Estrazione` / `Извлечение` for the extraction section, and each file's own wording for "disabled", "citation" and "anti-hallucination" as they appear in the neighbouring settings. The placeholder examples keep the same shape as the English source (main characters, game modes, weapons, battle pass), so a reader sees a concrete domain rather than a transliteration. Two things this commit does not do: - It does not touch the `lintTagViolation*` cluster, which is untranslated in the same eight locales and predates this feature. That is a separate change with its own PR, because a fix that bundles an unrelated gap cannot be reviewed on its own terms. - It does not add a guard that would have caught the original gap. The parity test checks that every key EXISTS in every locale; it cannot see that a key holds the English text. A guard is proposed separately, with the measurement that shows what it can and cannot flag. Gate 1: lint 0, tsc 0, build clean, tests pass, css-lint 0.
|
Follow-up: the i18n gap I flagged in the PR body is now closed. 56 strings — 7 keys × 8 locales — for Japanese, Korean, German, French, Spanish, Portuguese, Italian and Russian. Chinese and Traditional Chinese were already translated. The wording follows each locale's existing terms rather than a fresh glossary. The extraction section is Two things this does not doThe No guard test is added in this PR. The parity test checks that every key exists in every locale. It cannot see that a key holds the English text — which is exactly how the 56 strings above shipped with the feature and passed every check. A guard is worth adding, but the threshold has to be chosen against measurement rather than guessed: a blanket "must differ from English" rule would flag the ~5 % of strings that are legitimately identical across locales (language names, Gate 1: lint 0, tsc 0, build clean, 369 test files / 5,047 tests, css-lint 0. |
|
I ran this branch at The test uses the setting's own placeholder as the requirement ( It drives
1. A requested section is removed after generation
So the placeholder's own example cannot take effect. The setting's description also lists "structure" among the things it controls ( 2. Merges run without the requirement
|
A source note whose own filename is a machine identifier produced junk pages. When the ingested note is named `01988c6e_86e9_7b79_83b5_c8c4b354fdc9.md` or `202511042307.md`, the model sometimes extracted the source DOCUMENT itself as an entity and lifted the filename as its name — the result is `entities/<uuid>.md`, a page whose body merely re-narrates the summary it came from.
`isSourceSelfReference` rejects those names at normalization time, before a page exists:
- machine shapes are rejected regardless of what the source file is called — a UUID with `-`/`_` separators, a 12- or 14-digit timestamp, and any run of 13 or more digits optionally followed by a `_`/`.` plus a hex tail, which covers IM voice-note message IDs like `1921147873752416344` and snowflake-style keys;
- a verbatim copy of the source basename is rejected only when it carries a plugin-style `_` plus 6-or-more-hex-digit tail, so `TNF-α` extracted from `TNF-α.md` survives and a drama name from `庆余年_8a53fa.md` survives.
Phone-sized runs (11 digits or fewer) and years stay allowed, because those are legitimate names.
**The wiring is part of this PR, and that is the point.** The original commit added the predicate and its regexes but left the two `.filter(...)` calls that use them to the commit that followed. Taken alone it would have shipped an exported function that nothing calls, with tests that pass by calling the function directly — a guard that is not on the path. The diff shows the two filters.
The prompt-level layer ("do not extract the source document, do not use its filename as a name") is in the sibling PR #895, because it is a prompt change and this is a structural filter.
**Credit.** The two commits in this branch are @weqoocu's work and carry their author line. This squash names them; the merge commit's author is me, which is a GitHub property of squashing, not a judgement about who wrote the code.
Gate 1: lint 0, tsc 0, build clean, 367 test files / 5,038 tests, css-lint 0.
…nslated string
Six `lintTagViolation*` strings were English in eight of the eleven locales — 48 strings. A user in zh, zh-Hant, ja, ko, de, fr, es or pt pressed a button that said "Retag", and read a progress line in English while every neighbouring setting was in their own language.
| key | was |
|---|---|
| `lintTagViolationFiring` | `Retagging {current}/{total}: {path}` |
| `lintTagViolationFailed` | `Retag failed for {path}: {error}` |
| `lintTagViolationFixed` | `Retag complete. Fixed {fixed}/{total} page(s).` |
| `lintTagViolationFixedNone` | `Retag complete. No pages needed fixing (LLM kept current tags).` |
| `lintTagViolationItem` | `- [[{path}]] — invalid: {tags}` |
| `lintTagViolationRetagBtn` | `🏷️ Retag {count} page(s) with LLM` |
Italian and Russian had these translated. That is how the gap survived: two locales out of eleven were correct, and the check looked at the whole set.
## The guard that would have caught it
The existing parity test asks whether every key **exists** in every locale, because `getText()` falls back to English for a *missing* key. A locale that **copies** the English string is not a fallback — it is a file that reads as English on purpose, and every check passes.
The new test asks the other question: for every English prose string of at least 30 characters, does each non-English locale hold something different?
**The threshold is measured, not guessed.** At 30 characters exactly five strings across all eleven locales are legitimately identical to English — three format templates (`[{step}/{totalSteps}] {type}: {name}` and two lint-line templates, where the prose is a placeholder pattern and the wording lives in the substituted values) and two example tag-name lists (`person, organization, project, place`, which are sample tag NAMES rather than prose). Each is listed in the test with its reason, because an exemption without a reason is a hole with a comment.
At 40 characters the count drops to zero — but 40 misses `Retag failed for {path}: {error}` at 32 characters, which is one of the strings that actually shipped untranslated. So 30, with the exemptions the measurement showed are needed.
**What it catches, and what it cannot.** Of the prompt-customization gap in #895 it catches 5 of 7 keys. It misses the two headings under 20 characters, where an identical string is a weak signal rather than a proof. That limit is written into the test.
**The exemptions cannot rot.** A second test fails when an exemption is no longer needed anywhere, so the list cannot quietly become a place to hide new gaps. Its rule is "at least ONE locale still holds it verbatim" — `zh` writes `[{step}/{totalSteps}] {type}:{name}` with a full-width colon, which is punctuation localization, not translation. The first version of that rule was stricter and failed on exactly that string.
**RED, then GREEN.** With `ja` reverted to its English strings the guard failed and named all six keys; with them restored, 144 of 144 pass.
**What this does not claim.** It does not assert that every locale is fully translated — that claim is not true and this test does not make it. What it enforces is a measured floor: no English prose of 30 characters or more, with five documented exceptions.
Gate 1: lint 0, tsc 0, build clean, 367 test files / 5,044 tests, css-lint 0.
|
@KvikullSigurdur — both findings reproduce here. I ran them independently rather than taking the report as read, and they are correct. Reproduction (my own file, Mechanism, confirmed at the source:
Neither defect came from the split. The original branch had the same shape: The review gap was mine. The 14 tests in #895 assert that the template CONTAINS The two findings are different in kindFinding 2 is a wiring gap and should be fixed. The setting shapes a page at creation and is silently absent on every later write. A page built from five sources has the requirement in the first and not in the other four — a page that is inconsistent with itself, which is worse than the setting not existing. Finding 1 is a promise that the mechanism cannot keep, and that is a design decision, not a bug to patch. The section list is plugin-owned. So there are two coherent answers, and this needs a decision rather than a patch:
I lean towards A for this PR, with B recorded as a separate piece of work — but @weqoocu wrote the feature and the call is theirs. One question the report opens but does not write out: you mentioned "one design question about page types". It is not in the body, so I do not have it. If it is the A-versus-B question above, I have stated it. If it is something else — how a custom section interacts with the per-type label set, say — please write it out, because that is the part I would otherwise guess at. What is not blocking the release
Where this leaves the PRTwo of the three assertions in the original body are still mine to fix, and I will not merge this with the placeholder test as the only evidence:
The section-list question is @weqoocu's to answer, and the description text depends on it. I am holding the merge until that is settled, because the honest options produce different descriptions and I would rather write one final text than two. The reproduction test is on my branch, not this one, so this PR's CI stays green while the decision is open. Say the word and I push it here as the RED that drives the fix. |
…older
The 14 tests that came with this feature assert that the prompt template CONTAINS
{{content_requirements}} and that the rendered output does not contain the
literal placeholder. Both pass while the setting does nothing to a written page.
This is the effect test that should have existed: it drives the page factory
with a fake model client and asks what the file on disk contains.
RED on the current code, two failures:
1. A section the model wrote is gone from the file. stripUnknownSections drops
every heading outside SECTION_LABELS, so a section named in the setting's own
placeholder example cannot exist on the page.
2. A merge sends two prompts and neither carries the requirement:
[false, false] where [true, true] was expected.
Finding 2 is a wiring gap and this test drives its fix. Finding 1 is a
promise the mechanism cannot keep — the section list is plugin-owned — so it
waits on a decision about what the setting may promise.
Reported and independently reproduced by @KvikullSigurdur on the PR.
|
Sorry about the misleading page types question, it is something I had in mind but then edited out forgetting I had foreshadowed it. I started going over issues because I like the plugin a lot but would like a more granular control of the structure (sections) created for each note. I am not quite caught up enough on the architecture to weigh in on #873, but the idea came up that if this PR meant to introduce the feature of letting users define sections, it might be nice to control sections per page type. Events pages could have a timeline section, projects could have risks etc. |
The 14 tests that shipped with the feature assert that the prompt template
CONTAINS {{content_requirements}} and that the rendered output does not
contain the literal placeholder. Both pass while the setting does nothing to
a written page, so neither question is about effect.
This is the effect test: it drives the page factory with a recording fake
client and asks what the file on disk contains.
the creation prompt carries the requirement passes
the written page keeps the ## Risk Notes section fails
a merge carries the requirement [false,false] fails
Both failures are real, both were reported by @KvikullSigurdur, and both also
exist on the original #755 branch, so the split did not introduce them.
They are pinned with `it.fails` rather than committed red. A red assertion
records the defect once and then blocks every unrelated change; `it.fails`
records it, keeps the signal in the suite, and turns into a failure the
moment the behaviour is fixed, which is how the fix announces that the marker
has to come off. Verified that the trap fires: an `it.fails` around a passing
assertion reports "1 failed".
Finding 2, the merge path, is a wiring gap and its fix is agreed. Finding 1 is
a promise the mechanism cannot keep -- the section list is plugin-owned -- so
it waits on the decision in #902, which also carries @KvikullSigurdur's
per-page-type extension.
|
@KvikullSigurdur Your two findings reproduce here independently, and they are now pinned in the branch. I recorded the whole thing, your per-page-type idea included, in #902. The short version of why it is an issue and not a patch in this PR:
So your request is written down where it will actually be acted on. Please keep sending reports like this one. The reproduction and the file:line references made it cheap to confirm, and any other friction you run into is worth a note — issues and PR threads are both fine. |
Split out of #755. The code is @weqoocu's; the split, and the one removal described below, are mine. The commit carries their author line.
Two free-text fields that shape what the model looks for and writes about, without touching any contract the plugin owns:
extractionFocus— appended to the extraction prompt's Extraction Scope block as a domain-priority hint. It narrows what the model looks for; it does not replace the JSON protocol, the classification rules or the tag vocabulary.contentRequirements— appended to entity/concept page generation through a{{content_requirements}}placeholder after the merge-strategy block. It controls emphasis and structure; citation, wikilink, merge and anti-hallucination rules stay binding.Empty is the default, and empty adds nothing, so every vault that never configures these gets byte-identical prompts and keeps its prefix cache. Both fields are capped at 4,000 characters, so a runaway paste cannot bloat every batch.
What I removed, and the evidence
The submitted change also carried a per-topic override layer:
settings.wikiTopics,activeTopicId, aWikiTopicinterface, and a resolution path that preferred a topic's own fields over the global ones.It could not run. Nothing in the settings UI creates or edits a
wikiTopicsentry, and nothing elsewhere writes the field, so it was always[].activeTopicIdhad no reader and no writer. The only trace in the UI was a description string naming a scope the user could not reach.The key difference from "unused API surface": the submitted code was
activeTopic(settings, ...).find(...)— a branch that every prompt build evaluated and that could never be true. Shipping it means every future reader has to prove unreachability again before trusting the feature. The repository's rule for this is wire it or delete it before a release.This is a deletion of your work, @weqoocu, and it is reversible by adding the picker that writes the field. I have recorded it as an issue rather than dropping it silently; if you want the layer here instead, say so and the third PR goes up as-is with the picker alongside it.
i18n
Eight new keys in all eleven locales, key sets identical so the type barrel stays honest. Japanese, Korean, German, French, Spanish, Portuguese, Italian and Russian currently carry the English text; Chinese and Traditional Chinese are translated. That gap is real — it is called out here rather than left for a user to discover, and translating it is a separate, reviewable change.
Gate 4: Performance
trim+ length checks per prompt build. The clamp allocates only when the field exceeds 4,000 charsCredit, stated rather than assumed
If this merges by squash, the merge commit's author becomes the PR author — me — and @weqoocu's contribution would read as mine. A rebase merge keeps their author line. That decision belongs to the maintainer at merge time, not to me.
Gate 1: lint 0, tsc 0, build clean, 369 test files / 5,047 tests, css-lint 0.