Pass knowledge, endpoint and spec to a run without a config file - #140
Open
DavertMik wants to merge 9 commits into
Open
Pass knowledge, endpoint and spec to a run without a config file#140DavertMik wants to merge 9 commits into
DavertMik wants to merge 9 commits into
Conversation
--knowledge takes facts on the command line and keeps them in memory for
that run only, so credentials and one-off test data never land in
knowledge/. Plain text applies everywhere; frontmatter scopes it the way
a knowledge file does - url: for a page, endpoint: for an API endpoint -
and the rest of the file grammar follows, including wait/waitForElement
and ${env.VAR} interpolation. The flag repeats, since gray-matter reads
one frontmatter block per string.
KnowledgeTracker holds session entries beside the ones it loads from
disk and matches both through the same structural patterns. Its
constructor now takes an options object, which lets the API boat point
it at its own knowledge directory without a loaded web ConfigParser.
That directory was never read at runtime before, so endpoint knowledge
written by `api know` sat unused. Chief now loads it when planning an
endpoint and Curler when testing one, which is what makes --knowledge
useful for auth on the API side.
drill's --knowledge <path> becomes --save-knowledge <path>, in the CLI
and in the TUI. It saves what drilling learned rather than supplying
facts, and the two cannot share a flag on the same command.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
The flag was threaded through four option interfaces, four buildOptions functions and four addCommonOptions helpers to reach a tracker that already knew what to do with it. It now registers on the program the way --ws does: one option plus a preAction hook, both living in knowledge-tracker.ts beside the code that reads them. Commander merges parent options down through optsWithGlobals, so one registration covers every command, the mounted api/docs/prima subcommands included, and the flag can sit anywhere on the line - before the command, after it, or after variadic arguments. Prima, doc-collector and the api CLI go back to what they were; ExplorBot no longer carries a knowledge option at all. The constructor still takes one, which is what the tests use and what makes the module state a fallback rather than the only way in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run needs to know where the app is, what documentation describes it, and the facts an agent cannot infer. Only the last of those could be given on the command line, so the rest still needed a config file. API boat: --endpoint and --spec, mirroring EXPLORBOT_URL and EXPLORBOT_API_SPEC. --endpoint keeps its path prefix as api.baseEndpoint, which global mode used to flatten to the origin, and gives `api test` an endpoint it never had an argument for. A target passed as a full URL is stripped back to a base-relative path. prima: --spec names a Docbot application spec to read as page knowledge, through the new EXPLORBOT_SPEC variable, which resolves into dirs.spec in every config mode and so serves every browser command. Prima's env mirroring turns it into PRIMA_CLI_SPEC on its own. docs collect: --url gives a relative path argument its base URL. Global mode now materializes EXPLORBOT_KNOWLEDGE and EXPLORBOT_KNOWLEDGE_FILE into the site's knowledge directory, where they were inert before, so the variables reach exploration, prima, doc collection and API testing alike. Also fixes `config` printing an absolute dirs entry glued onto the project root. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEFNJs2DKN8jKwnzZpi7Dk
KnowledgeTracker answers what is true from what was stored; it has no business importing commander or owning a flag. The registration moves to src/commands/knowledge-option.ts, and the values it collects land in Stats, the session-state store the tracker already sits below. The tracker now reads Stats.knowledge and knows nothing about where it came from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEFNJs2DKN8jKwnzZpi7Dk
The option belongs where the run's inputs are already resolved, next to EXPLORBOT_KNOWLEDGE and materializeKnowledge, registered the way remote registers --ws. commands/ holds TUI command classes, not CLI wiring, so the file added there is gone and Stats no longer carries a value that was only passing through. KnowledgeTracker asks config for what the flag collected, through the dependency it already has. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEFNJs2DKN8jKwnzZpi7Dk
--ws and --knowledge belong to a run rather than to a command, and each was declared inside the module that consumed it: remote owned --ws, config owned --knowledge. src/commands/options/ now holds them as BaseOption subclasses — flags, description, an optional collect for a repeatable value, and an apply() that runs after parsing — with one public register(program) a bin calls. A bin registers what it offers, so prima keeps its surface and the others keep both flags. apply() hands the value on and stops there: --ws to remote.attach, --knowledge to config, which holds the run's inputs. remote loses its commander import and its command-path helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEFNJs2DKN8jKwnzZpi7Dk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--knowledgetakes facts on the command line and keeps them in memory for that run only, so credentials and one-off test data never land inknowledge/.npx explorbot explore /pay --knowledge 'my credit card is 13213213213'Plain text applies everywhere. Frontmatter scopes it the way a knowledge file does —
url:for a page,endpoint:for an API endpoint — and the rest of the file grammar follows, includingwait/waitForElementand${env.VAR}interpolation:The flag repeats, since gray-matter reads one frontmatter block per string, so several scoped facts need several flags.
How it works
It registers once on the program root, the way
--wsdoes, and both now live in the same place:src/commands/options/, oneBaseOptionsubclass per run-level flag —flags,description, an optionalcollectfor a repeatable value, and anapply()that runs after parsing, behind a publicregister(program)a bin calls. A bin registers what it offers, so prima keeps its surface while the others take both flags.apply()hands the value to its owner and stops:--wstoremote.attach(),--knowledgeto config, besideEXPLORBOT_KNOWLEDGEandmaterializeKnowledge, where the run's inputs are resolved whatever they arrive on.KnowledgeTrackerasks config for what the flag collected and knows nothing about a command line. Commander merges parent options down throughoptsWithGlobals, so that single registration covers every command — the mountedapi,docsandprimasubcommands included — and the flag can sit anywhere on the line: before the command, after it, or after variadic arguments. Nothing inExplorBot,Prima,DocBotorApiBotcarries a knowledge option; the three standalone boat bins get one registration line each.KnowledgeTrackerholds the parsed entries beside the ones it loads from disk and matches both through the same structural patterns — no new matching path. Its constructor takes an optional override, which is what the tests use and what keeps the module state a fallback rather than the only way in.Session entries stay out of
knowsandlistAllKnowledge, which report what is on disk. They do show in/context:knowledge, listed as--knowledge #1.The API boat now reads knowledge
knowledge/was never read at runtime by the API boat, so endpoint knowledge written byapi knowsat unused. Chief now loads it when planning an endpoint and Curler when testing one — which is what makes--knowledgeuseful for auth on the API side.This changes behaviour for existing users:
endpoint:files already on disk, andEXPLORBOT_KNOWLEDGE, now appear in Chief and Curler prompts where they previously did not.Breaking:
drill --knowledgerenamedexplorbot drill --knowledge <path>is now--save-knowledge <path>, in the CLI and in the TUI (/drill --save-knowledge). It saves what drilling learned rather than supplying facts, and the two cannot share a flag on the same command.Checks
bun test tests/unit(1084),tests/integration(80),boat/prima/tests(131),node --test tests/node/*.mjs(10), format and lint all pass. Eight new unit tests cover session parsing — no-frontmatter reaching every page and endpoint,url:andendpoint:scoping, several entries staying independent, frontmatter hints reachinggetStateParameters,${env.VAR}interpolation, nothing written to the knowledge directory — plus a round trip through the registered option proving the flag reaches a freshly built tracker.This one is worth a regression run before merge — it changes what reaches Chief and Curler prompts on every API run.
Also: endpoint and spec from the command line
The knowledge flag closes one of the three inputs a run needs. A config file was still the only way to say where the app is and what documentation describes it, so these commits close the other two in the same shape: a flag, an environment twin, and no file required.
API boat
explorbot api plan /users \ --endpoint https://api.example.com/v1 \ --spec ./openapi.yaml \ --knowledge 'Send X-Api-Key: ${env.API_KEY} on every request'--endpointsetsEXPLORBOT_URLand--specsetsEXPLORBOT_API_SPEC; both beat the config file when given. Two things this fixes beyond the convenience:https://api.example.com/v1becamehttps://api.example.com, so an API whose base carries a version or project segment could not sensibly be tested from~/.explorbot. The prefix is kept now, and a target passed as a full URL is stripped back to a base-relative path so it cannot double up.api testhad no way to name an endpoint. It takes a plan file, so in global mode it died inresolveSiteTargetunlessEXPLORBOT_URLhappened to be exported.prima
--specpoints at a Docbot application spec directory, or itsindex.md, to be read as page knowledge — the flag form of the--specthatexplorbot startalready takes. It works through a newEXPLORBOT_SPECvariable that resolves intodirs.specin every config mode, so every browser command gains it, and prima'sEXPLORBOT_*→PRIMA_CLI_*mirroring producesPRIMA_CLI_SPECwith no further code.docs collect
--urlgives a relative path argument its base URL:It travels as
baseUrlrather than through the environment, so it beats a URL pinned in a config file.Doc collection gets no
--spec: it is the command that writes one.Environment knowledge in global mode
materializeKnowledgeran only in config-free mode, soEXPLORBOT_KNOWLEDGEandEXPLORBOT_KNOWLEDGE_FILEdid nothing for a run on~/.explorbot/config.js. BothenterGlobalModeimplementations now write into the site's knowledge directory, so the two variables reach exploration, prima, doc collection and API testing alike. The semantics stay per-run: the next run rewrites the file, a run with neither variable removes it, andlearnandknowremain the way to keep a fact.Together with the knowledge flag, this is what lets a full API run happen with nothing on disk but the global model config.
Checks
bun test tests/unit(1134) andtests/integration(82) pass; format, lint andbunosh docs:sync --checkare clean. Seven new unit tests cover the endpoint prefix and its stripping, spec precedence over a config file, and environment knowledge materialized in both global modes. Each boat was also driven end to end against a sandboxedHOMEholding a global config: the API boat resolving endpoint, spec and knowledge together; prima loading an application spec throughEXPLORBOT_SPEC; doc collection registering a site from--url.One drive-by fix:
configprinted an absolutedirsentry glued onto the project root, visible as soon as a spec lived outside the project.🤖 Generated with Claude Code