Pass knowledge, endpoint and spec to a run without a config file

CHANGELOG.md

@@ -2,8 +2,49 @@
 
 ## 2026-08-30
 
+### New CLI Options
+
+- **`--endpoint`** (api) — The base API endpoint an `explorbot api` run tests, so the API boat no longer
+  needs a config file to know where the API is. `api test`, which takes a plan file rather than an
+  endpoint, reads it from here too, and a path prefix is kept: given
+  `https://api.example.com/v1`, a step on `/users` is sent to `https://api.example.com/v1/users`.
+  It sets the same value as `EXPLORBOT_URL`, and wins when both are given.
+  ```bash
+  explorbot api plan /users --endpoint https://api.example.com/v1
+  explorbot api test output/plans/users.md --endpoint https://api.example.com/v1
+  ```
+- **`--spec`** (api) — The OpenAPI spec for the run, as a local file or a URL. Chief plans from it and
+  Curler looks up schemas in it; given here it replaces `api.spec` from the config file. It sets the
+  same value as `EXPLORBOT_API_SPEC`.
+  ```bash
+  explorbot api plan /users --spec ./openapi.yaml
+  explorbot api plan /users --spec https://api.example.com/openapi.json
+  ```
+- **`--spec`** (prima) — The collected documentation a prima run reads as page knowledge: a Docbot
+  application spec directory, or its `index.md`. It is the flag form of the `--spec` that
+  `explorbot start` already takes, and the new `EXPLORBOT_SPEC` variable sets the same thing for
+  every browser command — `PRIMA_CLI_SPEC` for prima, like the other variables it mirrors.
+  ```bash
+  prima check "a project can be archived" --spec output/docs
+  PRIMA_CLI_SPEC=output/docs prima do "open the account menu"
+  ```
+- **`--url`** (docs collect) — The base URL to document when the path argument is relative, so the
+  site can come from the command line rather than only from an absolute path or the environment. An
+  absolute path argument still carries its own. Same value as `EXPLORBOT_URL`.
+  ```bash
+  explorbot docs collect /dashboard --url https://app.example.com
+  ```
+
 ### Changes
 
+- Knowledge from `EXPLORBOT_KNOWLEDGE` and `EXPLORBOT_KNOWLEDGE_FILE` now reaches runs that use the
+  global configuration in `~/.explorbot` — exploration, prima, doc collection and API testing alike.
+  Each run writes what they carry into the site's knowledge directory, where the agents read it like
+  any other knowledge file; the next run rewrites it, and a run that sets neither variable removes
+  it, so `learn` and `know` remain the way to keep a fact. Until now those two variables only had an
+  effect when no configuration file existed at all.
+- `config` no longer prints a directory as the project root with an absolute path glued onto the
+  end. An absolute `dirs` entry, such as an application spec outside the project, is shown as it is.
 - Doc Collector: a `docs collect` run streamed with `--ws` now sends the spec index it generates as a
   `docs` frame — the file path and the full markdown of `docs/index.md` — so a listening UI can show
   the finished documentation the same way an exploration run streams its session report.
@@ -185,6 +226,35 @@
 
 ## 2026-08-23
 
+### New CLI Options
+
+- **`--knowledge`** — Facts for one run, passed on the command line instead of stored in `knowledge/`.
+  Nothing is written to disk, so credentials and one-off test data stay out of the repository. Plain
+  text applies everywhere; frontmatter scopes it to a page (`url:`) or an API endpoint (`endpoint:`),
+  with the same patterns knowledge files use. `${env.VAR}` interpolation and page automation fields
+  such as `wait` work as they do in files. Repeat the flag for several facts. Like `--ws`, it is a
+  program-level option: it works on every command of `explorbot`, `explorbot api`, `explorbot docs`
+  and `prima`, and can go anywhere on the line.
+  ```bash
+  explorbot explore /pay --knowledge 'My credit card is 4111 1111 1111 1111'
+  explorbot explore / --knowledge '---
+  url: /login
+  ---
+  Log in as admin@example.com / secret123'
+  explorbot api explore /orders --knowledge 'Send X-Api-Key on every request'
+  prima check "checkout completes" --knowledge 'Use the sandbox card 4111 1111 1111 1111'
+  ```
+- **`--save-knowledge`** (renamed) — `explorbot drill --knowledge <path>` is now
+  `explorbot drill --save-knowledge <path>`, and `/drill --knowledge` is now `/drill --save-knowledge`.
+  It still saves the interactions drilling learned to a knowledge file at that URL path; the rename
+  frees `--knowledge` for the session facts above.
+  ```bash
+  explorbot drill /login --save-knowledge /login
+  ```
+  ```
+  /drill --save-knowledge /login --max-components 10
+  ```
+
 ### Changes
 
 - Prima reads `PRIMA_CLI_*` environment variables. Each one mirrors the `EXPLORBOT_*` variable of the
@@ -212,6 +282,10 @@
   the test.
 - [Pilot] The Pilot no longer pushes a full page of HTML into the Tester mid-test. It can still
   attach the accessibility tree, a page summary, or the UI map when recent actions failed.
+- [Chief] Now reads endpoint knowledge when planning API tests, so auth rules and business
+  constraints written with `explorbot api know` reach the plan.
+- [Curler] Now reads endpoint knowledge when running an API test, so auth headers and payload rules
+  reach the requests themselves rather than only the plan.
 - Prima now ships as its own npm package, so `npx prima-cli` runs it without installing explorbot
   first. It is the same tool as the `prima` command that comes with explorbot, built from the same
   source and released alongside it — only the package name and the binary differ.

CLAUDE.md

@@ -99,7 +99,7 @@ Two tiers. **Data parts** are deterministic memory — they answer "what is true
 | Module | Does | Must never do |
 |---|---|---|
 | **StateManager** | Where am I / where have I been: states, transitions, hashes, loop detection, events | Call AI; interpret semantics ("this is a login page"); execute actions |
-| **KnowledgeTracker** | Load/filter human facts by URL pattern; expose hints verbatim | Judge relevance semantically; act on hints; be written mid-run outside learn/drill flow |
+| **KnowledgeTracker** | Load/filter human facts by URL or endpoint pattern, from `knowledge/` and from `--knowledge` session entries; expose hints verbatim | Judge relevance semantically; act on hints; be written mid-run outside learn/drill flow |
 | **ExperienceTracker** | Store/retrieve lessons per state hash; dedup blocks | Rank by meaning; compact itself (ExperienceCompactor's job); change behavior directly |
 | **Config** | Resolve immutable settings once at startup | Change mid-run; hold site-specific values; be read by agents directly (injected instead) |
 
@@ -136,7 +136,7 @@ All persisted formats share one rule: **envelope keys (YAML frontmatter, HTML co
 
 | Format | Location & owner | Envelope | Body grammar |
 |---|---|---|---|
-| Knowledge | `knowledge/*.md`, KnowledgeTracker | `url`/`path`, `wait`, `waitForElement`, `noExperienceReading/Writing` | Free prose facts |
+| Knowledge | `knowledge/*.md`, KnowledgeTracker | `url`/`path`, `endpoint`, `wait`, `waitForElement`, `noExperienceReading/Writing` | Free prose facts |
 | Experience | `experience/<stateHash>.md`, ExperienceTracker | sparse frontmatter | `## FLOW:` / `## ACTION:` h2 blocks; bullets + ```js``` + `Solution:` line; h3 forbidden under blocks |
 | Test plan | `output/plans/*.md`, test-plan-markdown.ts | `<!-- test ... -->` comment: `priority`, `style`; scenario heading, `url:` line, bullets as steps | Notes/results appended by runner |
 
@@ -539,7 +539,7 @@ There are application commands available in TUI
 * /research [uri] - performs research on a current page or navigate to [uri] if uri is provided
 * /plan <feature> - plan testing feature starting from current page
 * /navigate <uri_or_state> - move to other page. Use AI to complete navigation
-* /drill [--knowledge <path>] [--max-components <n>] - drill all components on page to learn interactions
+* /drill [--save-knowledge <path>] [--max-components <n>] - drill all components on page to learn interactions
 
 There are also CodeceptJS commands available:
 
@@ -563,7 +563,15 @@ Consequently `isInteractive()` (`src/ai/task-agent.ts`) is **`INK_RUNNING || exe
 
 **There are no frame types.** A frame is `{type, ts, ...whatever}`; `send(type, data)` puts data on the wire and the UI renders what it recognises. Neither side validates the other's shape, so either can start sending more at any time. It queues while disconnected, reconnects with backoff, and `remote.close(exitCode)` flushes before exit (called from `showStatsAndExit`).
 
-`remote.registerOption(program)` adds the flag through a Commander `preAction` hook, so it covers every command including the mounted `api`/`docs` subcommands and the standalone `boat/*` bins.
+`--ws` itself is declared outside remote, as a `BaseOption` in `src/commands/options/` — see below — and its hook calls `remote.attach()`.
+
+## Shared CLI options (`src/commands/options/`)
+
+A flag that belongs to a run rather than to one command lives here, one `BaseOption` subclass per flag: `flags`, `description`, an optional `collect` for a repeatable value, and an `apply()` that runs after parsing. `register(program)` declares the flag on the program root and hooks `preAction`, so one registration covers every command, the mounted `api`/`docs` subcommands and the standalone `boat/*` bins alike, and the flag can sit anywhere on the line.
+
+Each bin registers the options it offers — `wsOption.register(program)`, `knowledgeOption.register(program)` — so a boat carries only the flags that mean something for it.
+
+`apply()` hands the value to whoever owns it and nothing else: `--ws` to `remote.attach()`, `--knowledge` to config, which holds the run's inputs. Nothing downstream learns that a command line exists.
 
 ## Command Line Usage
 
@@ -593,7 +601,7 @@ explorbot plan /login authentication  # plan with focus on authentication
 ```bash
 explorbot drill <url>                    # drill all components on page
 explorbot drill /components --max-components 10  # limit to 10 components
-explorbot drill /login --knowledge /login  # save to knowledge file
+explorbot drill /login --save-knowledge /login  # save to knowledge file
 ```
 
 ### Show resolved configuration:

bin/explorbot-cli.ts

@@ -10,6 +10,7 @@ import React from 'react';
 import { flushTelemetry } from '../src/ai/provider.js';
 import { App } from '../src/components/App.js';
 import { StatusPane } from '../src/components/StatusPane.js';
+import { knowledgeOption, wsOption } from '../src/commands/options/index.js';
 import { ConfigParser, EXPLORBOT_ENV_VARS, PROVIDERS } from '../src/config.js';
 import { ExplorBot, type ExplorBotOptions } from '../src/explorbot.js';
 import { remote } from '../src/remote.js';
@@ -28,7 +29,8 @@ const pkgPath = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../p
 const pkgVersion = JSON.parse(fs.readFileSync(pkgPath, 'utf-8')).version as string;
 
 program.name(cli).description('AI-powered web exploration tool').version(pkgVersion, '-V, --version');
-remote.registerOption(program);
+wsOption.register(program);
+knowledgeOption.register(program);
 
 process.on('uncaughtException', async (error) => {
   tag('error').log(`Uncaught exception: ${error instanceof Error ? `${error.message}\n${error.stack}` : String(error)}`);
@@ -701,7 +703,7 @@ addCommonOptions(program.command('navigate <url>').description('Navigate to a UR
 });
 
 addCommonOptions(
-  program.command('drill <url>').alias('driller').description('Drill all components on a page to learn interactions').option('--knowledge <path>', 'Save learned interactions to knowledge file at this URL path').option('--max-components <count>', 'Maximum number of components to drill')
+  program.command('drill <url>').alias('driller').description('Drill all components on a page to learn interactions').option('--save-knowledge <path>', 'Save learned interactions to knowledge file at this URL path').option('--max-components <count>', 'Maximum number of components to drill')
 ).action(async (url, options) => {
   try {
     const explorBot = new ExplorBot(buildExplorBotOptions(url, options));
@@ -710,7 +712,7 @@ addCommonOptions(
     await explorBot.visit(url);
 
     const plan = await explorBot.agentDriller().drill({
-      knowledgePath: options.knowledge,
+      knowledgePath: options.saveKnowledge,
       maxComponents: Number.parseInt(options.maxComponents || '30', 10),
       interactive: false,
     });

boat/api-tester/bin/apibot-cli.ts

@@ -1,7 +1,8 @@
 #!/usr/bin/env bun
-import { remote } from '../../../src/remote.ts';
+import { knowledgeOption, wsOption } from '../../../src/commands/options/index.ts';
 import { createApiCommands } from '../src/cli.ts';
 
 const program = createApiCommands('apibot');
-remote.registerOption(program);
+wsOption.register(program);
+knowledgeOption.register(program);
 program.parse();

boat/api-tester/src/ai/chief.ts

@@ -3,6 +3,7 @@ import { z } from 'zod';
 import { Conversation } from '../../../../src/ai/conversation.ts';
 import { WithSessionDedup } from '../../../../src/ai/planner/session-dedup.ts';
 import type { AIProvider } from '../../../../src/ai/provider.ts';
+import type { KnowledgeTracker } from '../../../../src/knowledge-tracker.ts';
 import { Observability } from '../../../../src/observability.ts';
 import { Plan, Test } from '../../../../src/test-plan.ts';
 import { createDebug, tag } from '../../../../src/utils/logger.ts';
@@ -33,17 +34,19 @@ export class Chief extends ChiefBase {
   private provider: AIProvider;
   private config: ApibotConfig;
   private apiClient: ApiClient | null;
+  private knowledgeTracker?: KnowledgeTracker;
   currentPlan: Plan | null = null;
   private lastStyleName = '';
 
   MIN_TASKS = 3;
   MAX_TASKS = 10;
 
-  constructor(provider: AIProvider, config: ApibotConfig, apiClient?: ApiClient | null) {
+  constructor(provider: AIProvider, config: ApibotConfig, apiClient?: ApiClient | null, knowledgeTracker?: KnowledgeTracker) {
     super();
     this.provider = provider;
     this.config = config;
     this.apiClient = apiClient || null;
+    this.knowledgeTracker = knowledgeTracker;
   }
 
   async plan(endpoint: string, opts?: { style?: string; specDefinition?: string }): Promise<Plan> {
@@ -56,6 +59,9 @@ export class Chief extends ChiefBase {
       const sampleData = await this.collectSampleData(endpoint);
       const conversation = this.buildConversation(endpoint, opts?.style, sampleData);
 
+      const knowledge = this.knowledgeTracker?.renderEndpointKnowledge(endpoint);
+      if (knowledge) conversation.addUserText(knowledge);
+
       if (opts?.specDefinition) {
         conversation.addUserText(dedent`
           <api_spec>

boat/api-tester/src/ai/curler.ts

@@ -2,6 +2,7 @@ import dedent from 'dedent';
 import { z } from 'zod';
 import type { AIProvider } from '../../../../src/ai/provider.ts';
 import type { RequestStore } from '../../../../src/api/request-store.ts';
+import type { KnowledgeTracker } from '../../../../src/knowledge-tracker.ts';
 import type { Reporter } from '../../../../src/reporter.ts';
 import { type Test, TestResult } from '../../../../src/test-plan.ts';
 import { createDebug, tag } from '../../../../src/utils/logger.ts';
@@ -18,12 +19,14 @@ export class Curler {
   private apiClient: ApiClient;
   private requestState: RequestStore;
   private reporter: Reporter;
+  private knowledgeTracker?: KnowledgeTracker;
 
-  constructor(provider: AIProvider, apiClient: ApiClient, requestState: RequestStore, reporter: Reporter) {
+  constructor(provider: AIProvider, apiClient: ApiClient, requestState: RequestStore, reporter: Reporter, knowledgeTracker?: KnowledgeTracker) {
     this.provider = provider;
     this.apiClient = apiClient;
     this.requestState = requestState;
     this.reporter = reporter;
+    this.knowledgeTracker = knowledgeTracker;
   }
 
   async test(test: Test, opts?: { specDefinition?: string; baseEndpoint?: string; searchSpec?: (query: string) => string }): Promise<{ success: boolean }> {
@@ -37,6 +40,9 @@ export class Curler {
     const conversation = this.provider.startConversation(this.buildSystemPrompt(), 'curler', this.provider.getAgenticModel('curler'));
     const tools = createCurlerTools(this.apiClient, this.requestState, test, opts?.searchSpec);
 
+    const knowledge = test.startUrl && this.knowledgeTracker?.renderEndpointKnowledge(test.startUrl);
+    if (knowledge) conversation.addUserText(knowledge);
+
     const initialPrompt = this.buildTestPrompt(test, opts?.specDefinition, opts?.baseEndpoint);
     conversation.addUserText(initialPrompt);
 

boat/api-tester/src/apibot.ts

@@ -3,6 +3,7 @@ import path from 'node:path';
 import { AIProvider } from '../../../src/ai/provider.ts';
 import { RequestStore } from '../../../src/api/request-store.ts';
 import { extractEndpointDefinition, loadSpec, searchEndpoints, validateSpecs } from '../../../src/api/spec-reader.ts';
+import { KnowledgeTracker } from '../../../src/knowledge-tracker.ts';
 import { Reporter } from '../../../src/reporter.ts';
 import { Plan } from '../../../src/test-plan.ts';
 import { setVerboseMode, tag } from '../../../src/utils/logger.ts';
@@ -20,6 +21,7 @@ export class ApiBot {
   private apiClient!: ApiClient;
   private requestState!: RequestStore;
   private reporter!: Reporter;
+  private knowledgeTracker!: KnowledgeTracker;
   private options: ApibotOptions;
   private apiSpec: any;
 
@@ -33,7 +35,7 @@ export class ApiBot {
   }
 
   async start(): Promise<void> {
-    this.config = await this.configParser.loadConfig({ config: this.options.config, path: this.options.path, endpoint: this.options.endpoint });
+    this.config = await this.configParser.loadConfig(this.options);
     this.provider = new AIProvider(this.config.ai);
     await this.provider.validateConnection();
 
@@ -47,6 +49,7 @@ export class ApiBot {
     this.configParser.ensureDirectory(outputDir);
     this.requestState = new RequestStore(outputDir);
     this.reporter = new Reporter(this.config.reporter);
+    this.knowledgeTracker = new KnowledgeTracker({ knowledgeDir: this.configParser.getKnowledgeDir() });
 
     validateSpecs(this.config.api.spec);
     this.apiSpec = await loadSpec(this.config.api.spec!, outputDir);
@@ -84,21 +87,22 @@ export class ApiBot {
     await this.apiClient?.teardown();
   }
 
-  createAgent<T>(factory: (deps: { ai: AIProvider; config: ApibotConfig; apiClient: ApiClient; requestState: RequestStore }) => T): T {
+  createAgent<T>(factory: (deps: { ai: AIProvider; config: ApibotConfig; apiClient: ApiClient; requestState: RequestStore; knowledge: KnowledgeTracker }) => T): T {
     return factory({
       ai: this.provider,
       config: this.config,
       apiClient: this.apiClient,
       requestState: this.requestState,
+      knowledge: this.knowledgeTracker,
     });
   }
 
   agentChief(): Chief {
-    return (this.agents.chief ||= this.createAgent(({ ai, config, apiClient }) => new Chief(ai, config, apiClient)));
+    return (this.agents.chief ||= this.createAgent(({ ai, config, apiClient, knowledge }) => new Chief(ai, config, apiClient, knowledge)));
   }
 
   agentCurler(): Curler {
-    return (this.agents.curler ||= this.createAgent(({ ai, apiClient, requestState }) => new Curler(ai, apiClient, requestState, this.reporter)));
+    return (this.agents.curler ||= this.createAgent(({ ai, apiClient, requestState, knowledge }) => new Curler(ai, apiClient, requestState, this.reporter, knowledge)));
   }
 
   async plan(target: string, opts: { style?: string; fresh?: boolean } = {}): Promise<Plan> {
@@ -200,6 +204,8 @@ interface ApibotOptions {
   config?: string;
   path?: string;
   endpoint?: string;
+  baseEndpoint?: string;
+  spec?: string;
 }
 
 export type { ApibotOptions };