Repository navigation
fix(gui): make the CTI status banner dismissable and check every request - #38
Merged
Merged
Conversation
Adds api_base support so the MCP plugin can target an OpenAI-compatible
gateway (e.g. AIP) instead of api.openai.com. Includes ${VAR} expansion
in default.yml so credentials live in the environment, not the file.
When yaml configures api_base + model, the yaml model wins over any GUI
submission — the gateway has a constrained model list, and stale
localStorage defaults like "gpt-4o" would otherwise produce a 401.
Backend changes flow api_base end-to-end: factory + planner clients,
the subprocess DSPY_API_BASE env, and the global dspy.configure path
in mcp_svc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an Enabled MCP Servers checkbox list to the Global Model Config panel. On mount, fetches /plugin/mcp/servers, defaults checked state to each server's default_enabled, persists picks to localStorage, and ships enabled_servers in the factory and planner POST payloads so the backend actually spawns the chosen subset. Without this, every run silently fell back to caldera_core only, so plugin-provided servers like RANGE were never reachable from the UI. Also bumps the form's default model to openai/nemotron-3-super to match the new yaml default; backend has a yaml-authority rescue for existing localStorage caches stuck on gpt-4o. Requires a magma rebuild (npm run build) for the bundle to pick this up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three layers, runnable independently with pytest:
1. discovery — server_registry picks up range/mcp_server.py and
parses MCP_METADATA via ast.literal_eval
2. spawn — range/mcp_server.py boots over stdio and exposes
range_-prefixed tools (no name collisions)
3. multi-merge — mcp_factory_client.run() spawns both servers and
merges tools without collision; LLM call is stubbed
so no API key is needed
Catches regressions where discovery breaks, range tools lose their
namespace prefix, or the multi-session merge stops working.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related observability fixes that together unblock the Ability Factory trajectory in MLflow. mcp.dspy.autolog() was commented out in the factory client, so every factory run completed without writing thought_/tool_name_/observation_ tags. The /plugin/mcp/status route reads exactly those tags, so the GUI saw an empty trajectory and polled forever waiting for it to fill in. Re-enabling it brings factory parity with the planner client. mcp_svc._run_execution() also called dspy.configure(lm=...) in a worker async task, which raised "can only be called from the same async task that called it first" the moment a second request landed. The call was caught in try/except, so it never crashed — but it was redundant: both clients already wrap react.acall in dspy.context(lm=...) which is task-local. Removed the configure block (and now-dead _yaml_llm import) so the warning stops printing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds /plugin/mcp/defaults so the UI can pull yaml-resolved config (model, api_base, temperature, max_tokens, max_tool_calls, rag_embed_model, rag_topk) from a single source instead of duplicating hard-coded defaults in mcp.vue. api_key is always returned blank — the user supplies their own. Fields the yaml doesn't (yet) name come from a small _CONFIG_FALLBACKS dict so adding a new yaml key automatically flows through. Wires /plugin/mcp/history/runs and /plugin/mcp/history/run — the mcp_history.vue tab calls these endpoints and the handlers (list_runs, get_run_detail) already existed in mcp_api, but no route was registered so the History UI silently failed. Removes the unused setup_routes() function in mcp_api.py — hook.py registers all routes inline; setup_routes was never called. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stops hard-coding model, temperature, max_tool_calls, max_tokens, ragEmbedModel, and ragTopK in mcp.vue. On mount, fetches /plugin/mcp/defaults and fills any field the user hasn't already set via localStorage. This kills the drift class that produced the "GUI=gpt-4o vs yaml=openai/nemotron-3-super" 401 — there is now one source of truth (conf/default.yml plus _CONFIG_FALLBACKS in mcp_api), and the UI reflects it. User overrides via localStorage still win; the fetch only fills blanks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reorganize the flat app/ directory into the layered structure documented in REFACTOR_PLAN.md. Pure mechanical moves with no logic changes; all imports updated, syntax checks pass, integration tests still pass. app/mcp_factory_client.py -> app/workflows/author.py app/mcp_planner_client.py -> app/workflows/plan_execute.py app/server_registry.py -> app/discovery/servers.py app/rag.py -> app/capabilities/rag.py app/factory.py -> app/dspy_env.py The caldera_core extension half (app/mcp_server.py and app/mcp_tools/) stays at the top of app/ deliberately. Putting it under app/mcp/ would create the path doubling plugins.mcp.app.mcp.tools, which only affects the plugin literally named mcp. Updated imports in hook.py, mcp_svc.py, mcp_api.py, mcp_server.py, and the integration test file. Updated the in-test fallback path in workflows/author.py and workflows/plan_execute.py so mcp_server.py is located one directory up from the new workflows/ folder.
Defines the two extension contracts the discovery layer will use to register plugin contributions. No registry consumers yet; this is pure type definition so future commits can land incrementally. Workflow ties an agent persona (DSPy signature) to its server scope, the capability ids it accepts, an optional plan validator dotted path, and a UI component path. The plan_validator field reserves space for the plan-then-execute pattern that pairs LLM inference with a deterministic spec-to-call translator on the tool-owning plugin side. Capability defines an orthogonal modifier with an enrich() function that runs before the workflow's agent loop and contributes named input fields to the workflow's signature.
Adds WORKFLOWS lists at the bottom of both built-in workflow modules so the discovery layer can pick them up once it lands. The orchestrator still routes by the legacy ExecuteStyle string for now, so behavior is unchanged for users. Author requires caldera_core, accepts the rag capability, and forbids range tooling because its signature already says do not deploy infrastructure. Plan and Execute requires caldera_core, accepts range optionally so the user can opt into infrastructure tools when needed, and accepts rag so CTI reports can ground both operation planning and infrastructure provisioning.
Adds a CAPABILITIES list at the bottom of rag.py exposing the existing RAGService through the Capability contract. The enrich function loads selected bundles from plugins/mcp/data/, builds a RAGService, retrieves the most relevant chunks for the user's prompt, and returns a cti_context string. Workflows that accept the rag capability and have a cti_context input field will receive this content automatically once the orchestrator switches to the capability registry. Inlines a copy of the rag context formatter inside the capability so it no longer depends on workflow code; the duplicate inline formatters in workflows/author.py and workflows/plan_execute.py will be removed when the orchestrator switch lands. This declaration is dormant; the legacy _build_rag_service_from_files path in mcp_svc still wires RAG into runs directly so behavior is unchanged for users.
Adds discover_workflows() and discover_capabilities() that mirror the
existing discover_mcp_servers() pattern. They scan two sources each:
1. plugins/<other_plugin>/mcp/workflows.py and capabilities.py
(the new external-plugin contract)
2. plugins/mcp/app/workflows/*.py and capabilities/*.py
(built-in registrations)
Built-ins are discovered after externals so they win on id collision.
Unlike server discovery which uses ast.literal_eval to avoid running
plugin code, workflow and capability discovery have to import the
modules because the registrations reference live Python classes
(signatures) and async functions. Import failures are logged and
skipped rather than blocking the rest of the registry.
hook.enable() now runs all three discovery passes and hands the
registries to MCPService. The service stores them but does not consume
workflow_registry or capability_registry yet; the orchestrator switch
is the next commit. /plugin/mcp/servers continues to work unchanged.
Local verification with the current plugin tree returns:
servers: ['caldera_core', 'range']
workflows: ['author', 'plan_execute']
capabilities: ['rag']
The mcp_svc orchestrator now dispatches by workflow id from the
workflow_registry built at boot, instead of branching on the
ExecuteStyle enum. The new flow is:
1. Look up the workflow by id.
2. Filter enabled_servers against the workflow's required and
optional set; required servers are pinned on regardless.
3. Filter enabled_capabilities against the workflow's accepted set.
4. Run each capability's enrich() in order; merge their outputs
into a context dict.
5. Invoke workflow.run(prompt, lm_obj, **context) so capability
fields like cti_context land in the signature's input fields
by name.
Both workflow run() functions now accept cti_context as a kwarg and
absorb unknown capability outputs via **_extra_capability_context, so
adding a new capability does not require touching workflow code.
The legacy /execute payload (type plus model.config.rag_files) still
works. mcp_svc.execute() accepts both shapes and translates the
legacy one into the new one via _LEGACY_TYPE_MAP. The translation
keeps RAG settings working, including the implicit RAG behavior the
old UI relied on where attaching rag_files enabled RAG even when type
was plain factory or planner.
ExecuteStyle is removed. The legacy shim and its callers can be
deleted once the UI ships the new payload exclusively.
The dspy.context() per-run pattern in each workflow's run() is left
in place; we deliberately do not call dspy.configure() from the
orchestrator (it raises across async tasks).
Adds two new GET routes that the UI uses to render itself dynamically
from discovery instead of hardcoding tabs:
GET /plugin/mcp/workflows
Returns each discovered workflow with its display name,
description, required and optional servers, accepted
capabilities, ui_component path, example prompts, and the
plan_validator dotted path (when set). Workflows whose required
servers are not all present in the discovered server registry
are filtered out, so for example a workflow that requires the
RANGE plugin disappears from the list when RANGE is not
installed.
GET /plugin/mcp/capabilities
Returns each discovered capability with its display name,
description, ui_settings_component path, and the input field
names it contributes.
Both endpoints read directly off MCPService.workflow_registry and
MCPService.capability_registry, which are populated once at boot by
the discovery layer.
mcp.vue is now data-driven: it fetches /plugin/mcp/workflows and
renders one card per discovered workflow instead of two hard-coded
"Factory" and "Planner" buttons. The card shows the workflow's
display name, description, and a one-line summary of which servers
and capabilities it can use. Adding a new workflow from any plugin
now results in a new card with no edits to mcp.vue.
The Global Model Config panel keeps only LM credentials and limits.
The "Enabled MCP Servers" checklist moves out of Global Model Config
and into each workflow's session page, scoped to that workflow's
required and optional servers. Required servers render disabled so
the user cannot turn them off. The "RAG TopK" and "RAG Embed Model"
fields move into the RAG capability's settings panel inside the
workflow page, since they only make sense when RAG is in play.
Workflow session pages now accept the workflow registration as a
prop. The two previously-distinct factory and planner pages collapse
into the same workflow shell that reads everything (title, examples,
allowed servers, accepted capabilities) from props.workflow. The
files keep their old names (local_mcp_ability_factory.vue and
public_mcp_ability_factory.vue) for now to keep the magma bundle
diff small; they can be renamed in a follow-on commit.
The /execute payload sent by the workflow shell switches to the new
shape: workflow_id, lm_config, enabled_servers, enabled_capabilities,
capability_settings. The legacy {type, config} shape is still
accepted by the backend, so a stale browser tab keeps working until
the user reloads.
The in-page Extension Guide is updated to teach the new contract:
mcp_server.py at plugin root, mcp/workflows.py and mcp/capabilities.py
inside an mcp/ folder, optional translator.py for plan-then-execute
workflows, and an optional Vue session page referenced by the
workflow's ui_component field.
localStorage shape gains three new keys (serversByWorkflow,
capabilitiesByWorkflow, capabilitySettings) so a user's per-workflow
choices and per-capability settings persist across reloads. RAG and
similar capability settings are stored globally rather than per
workflow, so picking a TopK once applies everywhere RAG is enabled.
…esign Captures the agreed shape for the MCP plugin redesign: three composable blocks (servers, workflows, capabilities) discovered at boot, the two-layer pattern for CTI to infrastructure (inference scaffolding plus spec-to-call translation), the new directory structure, plugin contracts, migration steps in order, and the open design questions. Documents the deliberate asymmetry where external plugins group MCP contributions under an mcp/ folder but the MCP plugin itself does not (to avoid the plugins.mcp.app.mcp.tools path collision).
The signatures' process_result OutputField descriptions previously read "Message that summarizes the result", which the model dutifully interpreted as a recap of which tools it called rather than the data those tools returned. Real symptom from a plan_execute run: process_result: "I first listed the available profiles to see what environments are configured. Then I retrieved the list of templates to know what deployment blueprints exist. After that, I fetched the VM images for each profile..." That tells the user nothing. The user wanted counts, names, and statuses, not a play-by-play of the agent's tool selection. The new descriptions explicitly forbid methodology narration and require the actual content (real names, ids, counts, statuses) be returned in a short list, with an exception for naming missing data when a tool failed. The WithRAG variants keep the secondary CTI note but subordinate it to the substantive results. Author and Plan and Execute share the same shape so they get the same fix in parallel.
Adds .env and .env.local to ignore so credentials never get tracked. Adds the plugin's planning and handoff markdown files to ignore so proposal documents stay local while implementation lands in code.
The plugin previously called load_dotenv() inside the caldera_core MCP subprocess, which meant the parent process never saw the file's contents and only worked because the subprocess re-loaded the same file. Move the load to hook.py so the parent reads .env at plugin enable; spawned subprocesses inherit the environment automatically. Adds python-dotenv to requirements.txt to back the load.
Adds app/config.py as the single place that merges yaml defaults, .env
credentials, and per-request UI overrides. Callers used to do this merge
inline in three different places with subtly different rules; the new
resolver makes the precedence explicit:
1. yaml defaults (model, api_base, ...)
2. env-resolved api_key (read from the env var named in api_key_env)
3. UI overrides (only non-empty fields, only fields not declared
fields_locked: true in yaml)
Renames the credential env vars so each one says what it is for:
CALDERA_API_KEY -> CORE_CALDERA_API_KEY (Caldera REST admin key)
yaml's ${API_KEY} -> MCP_LLM_API_KEY (LLM provider key, named
via api_key_env in yaml)
This is a breaking change for anyone with a stale .env. Update local
.env files to use the new names.
Restructures conf/default.yml to drop the dead factory: block and the
non-functional ${API_KEY} substitution, add a caldera: block declaring
which env vars to consult, and add a fields_locked map so deployment
constraints (gateway-fixed model lists) are explicit instead of silently
overriding UI input.
Updates .env.example to match. Updates the test that forwards env to a
range subprocess to use the new var name.
mcp_svc.execute() previously skipped DSPy client construction when the UI did not send an api_key, leaving the workflow to read yaml itself on the way down. The resolver now runs unconditionally for every request so the merge happens in one place: yaml defaults -> env credential -> UI overrides The rag capability still defaults its api_key to the resolved LLM key when the caller doesn't supply embed_api_key explicitly. mcp_api.defaults() switches to llm_defaults() from the shared config module, drops the dead workflows.author import, and now emits fields_locked so the UI can disable inputs that the deployment fixes (matching the resolver's behavior of dropping locked-field overrides).
Both workflow modules previously redefined get_llm_config and
_expand_env, then re-merged yaml with the caller's lm_obj inside
run(). When yaml had api_base set the merge silently overwrote the
UI's model field, and the override was invisible to the user.
mcp_svc now resolves the full LM config once via resolve_llm_config
(yaml + env + UI overrides, with fields_locked enforced). The
workflow's job is just to consume the resolved dict and build a
dspy.LM from it. Removes:
- _expand_env (the ${API_KEY} substitution it backed never worked)
- get_llm_config (replaced by config.llm_defaults for the test path)
- configure_llm (dead, never called)
- the build_lm_from_dict / yaml-overrides-UI block in plan_execute.py
- the lm_obj_safe / yaml_cfg merge block in author.py
The dspy.LM construction helper stays in plan_execute.py as
_build_lm_from_settings, callable from a single resolved dict.
The RAG capability previously hard-coupled embedding credentials to the chat LLM key. That breaks when the chat model lives behind a gateway with a constrained model list and the embedding model is hosted somewhere else (e.g. chat on MITRE gateway, embeddings on OpenAI). RAGService now accepts an api_base alongside api_key and forwards both to dspy.Embedder. The capability's enrich() honors two new optional settings: embed_api_key override for the embedding key embed_api_base override for the embedding provider's base URL When neither is supplied, the resolved chat-LLM key continues to flow through unchanged so existing single-provider deployments work without configuration.
McpGUI used to be vestigial: the class was instantiated in hook.py but no route was registered, splash() was unrouted, and the jinja2 template was the scaffold's Alpine.js click-the-button demo. Give the file a real job. splash() now assembles a readiness snapshot via the unified config resolver and the discovered registries: - Whether MCP_LLM_API_KEY is set (server-side default for LLM calls) - The model and api_base that requests will use - The Caldera REST URL the caldera_core subprocess will hit - Discovered MCP servers, workflows, and capabilities by name The template renders that snapshot as a status card and surfaces a warning when the LLM key is missing so the user is told up front that runs will fail until the .env or per-session UI key is supplied. Wires GET /plugin/mcp/gui to splash so the page actually renders when Caldera opens the plugin. The Vue SPA workspace stays the primary UI; this splash is the plugin's diagnostic / entry surface.
…ess halves
The single config.py file paired with the side-effect-import bootstrap
in dspy_env.py expressed two halves of one contract awkwardly. Promote
config to a package so the two halves are siblings under one
namespace:
app/config/__init__.py re-exports the parent-side public API so
existing imports keep working unchanged
(resolve_llm_config, llm_defaults,
caldera_connection)
app/config/parent.py the resolver code (was config.py). The
BaseWorld import is deferred into
_load_defaults so this module imports
cleanly even in subprocess contexts that
do not have Caldera's framework on
sys.path
app/config/subprocess.py the subprocess-side LM bootstrap (was the
top of dspy_env.py). Renamed from
configure_dspy_from_env to
ensure_lm_configured. Lazy and idempotent;
raises a clear RuntimeError when
DSPY_API_KEY is missing instead of
silently no-op'ing
dspy_env.py loses its module-level configure call. CreateCommand.__init__
calls ensure_lm_configured() so importing dspy_env is now inert; the
global LM state changes only when CreateCommand actually needs it. The
file's remaining job (signatures plus the CreateCommand module) is
unchanged.
The DSPY_* env-var names that bridge parent and subprocess are now
constants in config/subprocess.py, documented as the contract the
parent's get_env helper writes to.
The execute() and _run_execution() methods were broken up by ASCII section dividers (---- 1. ... ----, ---- 2. ... ----) that added visual noise without making the flow clearer. Remove the dividers and the comments that just described the next block; keep only the ones that explain a non-obvious decision (legacy payload mapping, deliberate dspy.configure absence, RAG default-key injection). No behavior change. Class init, signatures, and side effects are unchanged.
The config package introduced a __init__.py that re-exported the parent API, plus a deferred BaseWorld import inside _load_defaults to keep parent.py importable from subprocess context. Both were structural concessions to a layout that did not earn them: the package only ever held two siblings, and the subprocess never imported parent anyway. Flatten back to a single app/config.py. Move the BaseWorld import to the top of the file (it is the parent-side resolver; the subprocess does not import this module). Drop the __init__.py and the package directory entirely. Existing imports (from plugins.mcp.app.config import resolve_llm_config) keep working unchanged. The subprocess-side bootstrap moves into dspy_env.py in the next commit; this one only renames and rewrites the parent half. config_subprocess.py is staged for deletion in the next commit.
…with workflows
dspy_env.py is now the single owner of the DSPy library contract for
the caldera_core MCP subprocess:
- The DSPY_* env-var names that bridge parent and subprocess
- The lazy LM bootstrap (ensure_lm_configured) that runs on first
CreateCommand() instantiation
- The signatures and CreateCommand module mcp_server.py uses
The cross-module reach into config.subprocess is gone. dspy_env.py
imports only os and dspy at the top of the file. Importing it has
no side effects; DSPy is configured only when CreateCommand actually
needs it. Missing DSPY_API_KEY raises a clear RuntimeError naming
the parent-side env-forwarding contract.
The parent's get_env helper in workflows/author.py and
workflows/plan_execute.py now imports the ENV_* constants from
dspy_env, so the env-var contract has a single source of truth in
the file that authoritatively reads them.
Deletes app/config_subprocess.py (the in-between file). With this
commit the layout is two flat modules:
app/config.py parent-side stack and resolver
app/dspy_env.py subprocess-side DSPy library bootstrap
Both workflow views polled status into pollFinalResult but never rendered it. Reasoning and Thoughts displayed; the substantive answer the agent produced was dropped on the floor. Add a Result panel above Thoughts that shows pollFinalResult when the run is finished. Uses a violet accent matching the workflow theme to distinguish the answer from intermediate reasoning.
The workflows already return the agent's process_result, reasoning,
and trajectory back up the call chain to mcp_svc. mcp_svc was
discarding everything except a 250-char preview tag, and the API
was reaching back into MLflow to recover the same data the agent
had just returned in memory.
Add an OrderedDict run cache on MCPService keyed by run_id with an
LRU bound. _run_execution writes RUNNING on entry, FINISHED with
the agent's full payload on success, and FAILED with the error
message on exception. get_run(run_id) returns the snapshot for the
upcoming /status read switch.
Both workflows now return {process_result, reasoning, trajectory}
instead of just process_result so the cache has everything the UI
needs without scraping MLflow tags.
Every existing mlflow.set_tag and mlflow.log_param call stays in
place. MLflow keeps its observability role; this commit only adds
a parallel in-memory state path. The /status endpoint is switched
to read from the cache in the next commit.
The /status endpoint now calls mcp_svc.get_run(run_id) and returns
the cached snapshot directly. The agent's process_result, reasoning,
and trajectory travel agent -> workflow -> mcp_svc cache -> API
without round-tripping through MLflow's tag and param store.
Side effects:
- Field-name mismatches between writer and reader can no longer
cause the UI to silently render empty panels. There is one
source of truth shared by writer and reader.
- Polls hit an in-memory dict instead of an MlflowClient RPC.
- Returns 404 when the run_id is older than the LRU bound or the
server restarted mid-run; the History endpoints still surface
those runs from MLflow.
Drops the log_param('process_result', ...) band-aid that was added
in plan_execute and author so MLflow could ferry the answer back
to the API. The corresponding set_tag('process_result', ...) stays
so the answer is still visible in the MLflow UI for that run.
History endpoints (list_runs, get_run_detail) still read from
MLflow; that is the right boundary.
Both panels displayed these and both stored their own copy, so they drifted: CTI Extraction read 0 and 8192 while Global Model Config read 0.5 and 24000, with nothing to say which a run would use. They move to the llm profile, so there is a single value rather than two kept in step. Editing either panel edits the same key, and a cti override is dropped and logged like any other. default.yml sets them explicitly, which also settles the disagreement between app/config.py's numeric fallbacks and the llm_client wire defaults: both resolvers now read the same number. Returning from a sub-view re-reads /defaults, because this view is hidden rather than unmounted and the CTI panel can have changed either value while it was hidden. temperature ships 0.0, not the 0.5 the chat fallback used, because Stage 1 parses the model's own output as JSON and sampling makes that less reliable. top_p, timeout, stream, offline and embed_model stay per workload.
Every branch of the cleaner wrote <stem>.txt, so report.md and report.txt both became report.txt. The cleaner gathers files concurrently, so whichever finished second replaced the other, and an interleaved pair produced text belonging to neither, which is then what the extractor parsed. Everything downstream is keyed on that stem, so the two reports also collapsed onto one IR and one bundle. Normalising md to .txt introduced the txt/md pair; html and pdf could already collide with each other the same way. clean_stem folds the source extension in, so report.md becomes report_md.txt. Stage 1's selection filter and the clean-to-raw reverse lookup use the same helper, the latter having matched on a bare stem and so moved the wrong raw file to processed. Clean, IR and STIX artifacts are renamed by this. Existing ones are not migrated; they are re-derived on the next run.
finalize_run took the selection and never read it, so every upload was retired to raw/processed whether or not this run touched it. list_cti_raw then stamped a literal status on that directory, so a report that was never selected, never cleaned and never extracted rendered green. file_status, which computes the status from the IR and its mtime, was already written and never called. It is wired in for both directories and its IR path now goes through clean_stem. The unused dedup set is used.
The misplaced-key guard was scoped to workload profiles, so the llm section took any key at all and any new top-level section was created on demand. An Authorization header, an x-api-key and an extra_headers block all reached conf/local.yml in plaintext, beside tracked config, under a file header promising credentials never go there. mlflow.host reached it too, and hook.py passes that to 'mlflow server --host'. One save could rebind an unauthenticated server holding every logged prompt and response to a non-loopback interface. get_config grew a 'resolved' sibling, which broke the exact-key-set test in unwrap_config_envelope, so a GET-edit-POST round trip was no longer recognised as an envelope and slipped past the per-section rules entirely. Sections and keys are now allowlisted, the envelope tolerates its sibling, and the secret pattern covers header and token spellings. A credential-named key is still stripped rather than refused, so an older cached bundle that posts api_key can save the rest of its payload.
is_valid_actor called name.isalpha() on the whole string, which is False as soon as there is a space, so every two-word actor fell through: Berserk Bear, Wizard Spider, Fancy Bear and Cozy Bear included, two of which the function's own docstring says to keep. Digits are allowed now too, since FIN7 and TA505 are names, but a bare number is not. mcp_server had a bare 'from dspy_env import' above the sys.path insert meant to make it resolvable, so the module imported only when sys.path[0] happened to be app/. Four tests asserted things that are no longer true or never were: test_preserves_dots wanted normalize_phrase to keep dots in an IP, but the function is documented as canonical alphanumeric form and its only callers use it as a dedup key, where stripping punctuation is the point. The entity extraction that cared about IPs was removed earlier on this branch. test_slash_actor_split expected ALPHV/BlackCat to split into two actors. No slash handling exists anywhere in the module, and those are two names for one group, so splitting would invent a duplicate. test_server_class_imports and test_get_server imported MCPServer and _get_server. git log -S finds no commit that ever added either. They are replaced by guards on what the module does expose, including the import that was broken. test_cti_has_parameter_fields asserted the cti section declares temperature and max_tokens. Those live on llm now; it asserts the resolved profile.
Emitted a DeprecationWarning on every STIX bundle, observed-data object and uploaded file. Timezone-aware now, and the isoformat suffix stays Z rather than +00:00 so bundle timestamps keep the shape STIX expects.
convert_ir_to_stix reads attack_patterns, threat_actors and hashes. It has never had a branch for malware, tools or infrastructure, so every one of those was extracted, canonicalised, enriched with command lines, written to the IR and then dropped at conversion. The LLM prompt asked for all three, so this was also paid for in tokens on every report. Removed: the three fields from the prompt and the schema, the well-known tool scan, the spaCy NER tool pass, the regex infrastructure pass and the tool and malware halves of the MITRE entity index, plus the command extraction that existed only to attach commands to them. The infrastructure pass was the IP and domain regex that survived the earlier removal of entity extraction, which this branch dropped on the grounds that CTI says what to run and the environment says where. Verified by running the pipeline end to end offline before and after: the emitted bundle is identical once the wall-clock timestamps are normalised.
The blanket utcnow replacement in 91520a7 wrote datetime.now(datetime.UTC) into a module that does 'from datetime import datetime', where that attribute does not exist. hashes_to_stix_observed_data raised AttributeError on every report carrying a hash, and cti_pipeline_stage1 turns that into 'Stage 1 failed for all N files', so one hash took the whole batch down. Nothing crossed that seam: the function is reached only from convert_ir_to_stix when the IR carries hashes, and no test produced one. It has tests now. The object was also missing spec_version, first_observed and last_observed, so stix2.parse rejected it outright and no consumer could load a bundle that contained one. It round-trips through the reference parser now.
Both workflows built the subprocess environment with
str(lm_settings.get('temperature') or 0.5). 0.0 is falsy, and 0.0 is the
shipped value: conf/default.yml sets it because Stage 1 parses the model's own
output as JSON. So the one setting that exists to keep extraction
deterministic was the one silently replaced. max_tokens had the same shape,
and the ssl_verify and timeout lines beside them were already correct.
dspy_env._DEFAULTS held temperature 0.5 and max_tokens 10000 against
default.yml's 0.0 and 24000, a third resolver disagreeing with the two that
5fbc99d had just reconciled. A test now asserts they stay equal.
Also replaces the two utcfromtimestamp calls 91520a7 missed, in a file it
already touched.
Every symbol below was grepped across app/, gui/, tests/, conf/, docs/, mcp_server.py, hook.py and the markdown before removal. The only dynamic dispatch in the plugin is app/discovery reading module-level WORKFLOWS and CAPABILITIES attributes; MCP tools are explicit decorators and routes are explicit add_route calls, so nothing here was reachable by name. cti_linguistics.py goes from 401 lines to 66. It kept HASH_PATTERNS and extract_hashes, which Stage 1 uses, and lost behaviour normalisation, fuzzy canonicalisation, entity typing and the dynamic technique extraction, none of which had a caller. Its normalize_behavior_text was a second, diverging copy of the one cti_nlp_enhancements defines and actually uses. Also removed: the STIX open-vocabulary loader and its 75-line data file, the infrastructure type triggers, init_mlflow, build_dspy_lm, prepare_raw_uploads, a duplicate new_stix_id, _is_valid_cti_text, _caldera_root, two copies of an uncalled create_tool_function, five section helpers in the ability library loader, and the rag module-level shims, which tested a global that is defined nowhere and so could only ever return their placeholder string. Then the imports and locals those orphaned. pyflakes is clean across the plugin. Verified by running the pipeline end to end offline on a report carrying a file hash: both bundles parse with stix2.
test_set_config_with_different_models POSTed four model names in a loop to localhost:8888, so whichever ran last was left in the operator's conf/local.yml and in the server's config cache. That is how a deployment configured for Qwen ended up asking Featherless for meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 and taking a 404 on every Author run. The conftest guard restores the file but not the running server: set_config caches what it wrote, so the value kept being served from /defaults, the panel read it back, and the browser wrote it to disk again. Restoring a file cannot undo a write to a live process. Retargeting these from cti to llm in ecaf472 is what made it reach the connection at all; on cti the allowlist had been rejecting them. set_config has 37 offline tests that drive the real handler against tmp_path, so the four live classes plus two more in the other suites were duplicating that coverage while rewriting a running deployment. Removed. No live POST to set_config remains.
The README config example showed temperature and max_tokens under cti, which 5fbc99d moved to llm; layered_profile drops them and set_config returns 400. Its key list also disagreed with default.yml. Both now match WORKLOAD_OVERRIDABLE, asserted against the frozenset. The CTI page still advertised entity extraction, removed in d918d15.
It globbed data/outputs_stix, which is gitignored, so on a clean checkout the glob was empty and it printed and returned without asserting. Where it did find files it checked five objects and only spec_version on threat-actor. It now builds a bundle from our own builders and runs stix2.parse over it, plus spec_version, id, created and modified on every object. Verified by removing spec_version from a builder: the test fails.
Module scope killed whatever held the MLflow port, ran subprocess.Popen, then blocked up to 10s polling. So any import did that, including pytest collecting a module that imports hook. Moved to _ensure_mlflow_server(), called from enable(). Import now takes 3s and spawns nothing. kill_existing_mlflow is deleted. It ran only inside 'if not is_port_open', so nothing was ever listening to find, and it never checked the victim was MLflow before killing it.
A wrong model surfaced three different ways, none of which named the setting:
litellm said the model does not exist, Featherless said it is not available
for inference, and for the same model on another day said it was temporarily
at capacity. The CTI path dumped the provider's raw JSON.
LLMHTTPError carries the status and explains itself, and is_transient
separates a wrong setting from a blip. cti_parsing degrades to offline
extraction only when the failure is transient; a bad model or a rejected key
now fails loudly instead of silently producing a worse bundle.
summarize_exception gains the same rewrites for the DSPy path.
before: litellm.NotFoundError: Custom_openaiException - The model
`meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` does not exist.
after: Model not available: the provider has no
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8. Set a model it
serves in Global Model Config, or in llm.model in conf/local.yml.
Four tests POSTed to cti/run against a running server with step 'all'. An empty file list means everything, so they ran Stage 1 and Stage 2 over the operator's real corpus: a paid LLM call per report, and every bundle in outputs_stix re-minted with a new id. Each asserted only that the status was one of two or three values, so none could fail. Removed. Three uploads, pytest_upload.txt, pytest_report.html and pytest_listing_test.txt, were never deleted and accumulated in the real data/raw/uploads. An autouse fixture now sweeps them, alongside the existing conf/local.yml guard. Verified: conf/local.yml md5 and the 38 files under data/ are identical before and after a full run.
Popen's handle was discarded and Caldera has no plugin teardown hook, so the tracking server was reparented to init on shutdown and held the port across every restart. Observed on this machine: mlflow pid 8451, parent 1. atexit terminates it, then kills it if it ignores that. The handle is set only when this process actually spawned one, so a server the operator started themselves is left alone. Nothing can be done about SIGKILL.
The banner was one hardcoded is-info box that nothing ever cleared. All 12 messages landed in it, so a failed upload rendered in the same blue as a completed run, and whatever it last said stayed on screen for the rest of the session while the operator worked further down the page. It carries a tone now, which picks the Bulma class: info while work is in flight, success when an action finishes, warning for a nudge, danger for a failure. A success clears itself after six seconds. Everything else waits, because progress is replaced by its own outcome and a failure is the one message worth reading. Staging a new file also clears it, since the previous run's outcome no longer describes what is selected. The close button matches the one stixViewer.vue already uses, and the timer is cancelled on unmount so it cannot fire into a view that is gone. Closes mitre#35
Six of the eleven fetches in cti.vue never looked at the response at all,
and checking res.ok would not have saved them. Caldera gates plugin routes
per handler, and check_permissions answers an expired session by raising
HTTPFound('/login'); fetch follows that redirect, so the reply arrives as a
200 carrying the login page. res.ok is true, res.json() then throws on the
HTML, and the rejection reached nobody: both tables simply stayed empty with
nothing on screen to say why. Download was worse, saving the login page
under the bundle's filename, and get_stix opened an empty modal.
Every call now goes through request() or requestJson(), which reject with a
message written for the status banner. They inspect res.redirected for the
login page and treat 401 and 403 the same way, since a non-default login
handler refuses rather than redirecting. Failures report as red banners the
operator can read and dismiss, and a failed delete keeps the selection so it
can be retried without picking the files again.
The guards are static because the plugin ships no JavaScript test runner.
They assert the only fetch left is the one inside the helper, that the
helper still looks past the status code, and that the banner keeps its close
control and its outcome-driven class.
The banner was one hardcoded is-info box that nothing ever cleared. All 12 messages landed in it, so a failed upload rendered in the same blue as a completed run, and whatever it last said stayed on screen for the rest of the session while the operator worked further down the page. It carries a tone now, which picks the Bulma class: info while work is in flight, success when an action finishes, warning for a nudge, danger for a failure. A success clears itself after six seconds. Everything else waits, because progress is replaced by its own outcome and a failure is the one message worth reading. Staging a file retires the previous confirmation, but only a success and only once the file is really staged. A pipeline can run for five minutes with the banner as its sole progress indicator, and the failure the operator is staging a file to retry is the message they most need to keep. The close button matches the one stixViewer.vue already uses, and the timer is cancelled on unmount so it cannot fire into a view that is gone. Closes mitre#35
Six of the eleven fetches in cti.vue never looked at the response at all,
and checking res.ok would not have saved them. Caldera gates plugin routes
per handler, and check_permissions answers an expired session by raising
HTTPFound('/login'); fetch follows that redirect, so the reply arrives as a
200 carrying the login page. res.ok is true, res.json() then throws on the
HTML, and the rejection reached nobody: both tables simply stayed empty with
nothing on screen to say why. Download was worse, saving the login page
under the bundle's filename, and get_stix opened an empty modal.
Every call now goes through request() or requestJson(), which reject with a
message written for the status banner. They inspect res.redirected for the
login page and treat 401 and 403 the same way, since a non-default login
handler refuses rather than redirecting. Failures report as red banners the
operator can read and dismiss, and a failed delete keeps the selection so it
can be retried without picking the files again.
A download continues past a bundle that has gone from disk, since a 404 for
one file says nothing about the rest of the selection, and names what it
skipped at the end. Only an expired session stops the loop, because that
fails every remaining file the same way.
The list refreshes the poll loop fires may not bury the banner. They run on
the loop's schedule rather than the operator's, so a background refresh only
writes over an empty banner or a spent success: otherwise one transient error
replaced the running indicator for the rest of the run, or buried the
pipeline outcome the operator was waiting for.
The guards are static because the plugin ships no JavaScript test runner.
They assert the only fetch left is the one inside the helper, that the
helper still looks past the status code, and that the banner keeps its close
control and its outcome-driven class. They read the .vue file at import, so
an unrecognisable tag yields an empty string and fails a test, rather than
raising a collection error that would abort the entire pytest run.
…ix/cti-ui The remote branch holds an earlier version of these same two commits, pushed before the branch was reviewed. Both conflicts resolve to the local side, which is that same work plus the four defects the review found: - staging a file cleared the running indicator and unread failures - one missing bundle cancelled the rest of a download selection - a background list refresh could bury the pipeline outcome - the test module raised at import, aborting the whole pytest run Nothing from origin/main or upstream/main is involved: their gui/views/cti.vue is byte-identical to this branch's merge base, and tests/test_cti_view_requests.py does not exist there.
# Conflicts: # README.md # gui/views/cti.vue
The extraction model panel renders inside the CTI Ingest Pipeline page, and its save checked res.ok. An expired session is answered with a 302 to /login that fetch follows, so the reply arrives as a 200 carrying the login page: the panel showed a green "Saved", then emitted saved, which reloaded the config and raised the red "Session expired" banner beside it. The page contradicted itself, and the operator's settings were never written. request() and requestJson() move out of cti.vue into gui/composables so both files on the page share one checked path. cti.vue keeps its eleven call sites unchanged; the panel now reports what request() phrases, so an expired session reads as expired and a 400 states the reason the API gave. The guards follow the helper: no file on the CTI page may call fetch, both must import the shared module, and the panel may not go back to trusting res.ok.
The upload sweep covered raw/ and clean/ but not stix_cti, where the live suite POSTs through stix/upload. Five pytest_stix bundles had accumulated in the operator's data. outputs_stix and outputs_ir/complete are covered for the same reason.
request() rejects on the response, but reading the body is a second await that sat outside the try. A connection dropped mid-download therefore threw past the handler into a click listener with nothing to catch it: the rest of the selection was abandoned and the list of what failed was never shown. The whole per-file download moves inside the try, so a body that fails is recorded like any other refusal and the remaining bundles still save.
Six defects the branch review and the page audit turned up, all of which end with the operator believing something happened that did not. Selecting a folder and pressing Run Pipeline sent the folder name to cti_run, which takes bare filenames. Nothing inside was extracted, stage 2 re-emitted STIX from whatever IR was already on disk, the folder was moved out of uploads, and the page reported "Pipeline complete." in green. A folder and a nested child are both refused now, and named, since neither can be run. Run Pipeline stayed live during a run, and every poll chain reads one server-wide status slot, so a second run made both chains announce the first one's outcome and stop; the later run's failure reached no one. The button is held for the length of the run. Leaving the page orphaned the poll chain: it kept writing into a dead component for five minutes while the fresh instance never polled at all. The chain now carries a generation that unmount invalidates. The model panel accepted a cleared Temperature or Max Tokens, because '' is both >= 0 and <= 1, and wrote the blank into the shared llm profile, which has no empty-value guard. Both are required now, an out-of-range value says why Save is inert instead of silently disabling it, and an edit retires a stale "Saved" rather than letting it vouch for unsaved changes. Also from the review: a failed config reload no longer buries a running indicator, a suppressed background failure is logged rather than dropped, a failure banner is announced as an alert rather than politely, the download anchor is attached before it is clicked and its object URL revoked a tick later, and the test sweep recurses so outputs_stix/debug stops accumulating fixtures in the operator's data directory.
pollCtiStatus is started without await, so a throw inside it rejected a promise nobody held: the in-flight flag stayed set and Run Pipeline was disabled for the rest of the session. A null body from /cti/status is enough to trigger it. The call now carries a catch that clears the flag and reports, so an unexpected failure costs the run, not the button.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #35, plus a related class of bug I hit on the same page while I was in there.
The status banner was a single hardcoded blue box. Every message went into it, so a failed upload looked identical to a completed run, and whatever it last said stayed on screen until something happened to overwrite it. There was no way to close it. It now takes its colour from the outcome, has a close button like the one in
stixViewer.vue, and clears itself after a success.Fixing that turned up something worse. The page can't tell when a session has expired. CALDERA answers an unauthenticated request with a redirect to
/login, andfetchfollows redirects, so what actually comes back is a 200 with the login page in the body.res.okis true.res.json()then throws on the HTML and nothing catches it, which is why the tables just render empty with no error at all. Six of the eleven calls weren't checking the response; checkingres.okwouldn't have rescued the other five either. They all go throughgui/composables/request.jsnow.The model config panel had the same bug in a more annoying form, showing a green "Saved" for saves that never left the browser. It would also let you clear the Temperature field and save that, since
'' >= 0 && '' <= 1is true, and thellmprofile it writes into doesn't guard against empty values.The one I'd look at hardest is Run Pipeline. Tick a folder instead of a report and the folder name gets sent to
cti_run, which only takes bare filenames. Nothing matched, stage 2 re-emitted STIX from whatever IR was already on disk, the folder was moved intoprocessed/, and the page said "Pipeline complete." The button also stayed live mid-run, and since both poll chains read the same server-wide status slot, a second run made both of them announce the first one's result. Navigating away lost the outcome entirely.Type of change
How Has This Been Tested?
Against a running CALDERA: an unauthenticated request really does come back
302 -> /login, and following it the wayfetchdoes gives a 200 oftext/htmlthatres.json()chokes on. Checked in the browser too, for GET, POST and the STIX endpoints.Drove the changed functions against a stubbed server for the cases that matter: expired session, a 404 partway through a multi-bundle download, a body that dies mid-transfer, a list refresh failing during a run, and folder and nested selections. Also checked the poll chain stops once it's stale, and that Run Pipeline gets released whether the run settles normally or the chain throws.
Both SFCs compile with the project's own
@vue/compiler-sfc, and I exercised the page in a browser against a live server.tests/test_cti_view_requests.pyis new. They're static guards rather than component tests, since the plugin ships no JS test runner. I ran each one against a deliberately broken copy of the file to make sure it actually fails on the thing it claims to check.tests/conftest.pyalso sweeps recursively now, sooutputs_stix/debug/stops filling up with test fixtures.Checklist: