Skip to content

[Feat] Model connection diagnostics and first-run readiness #100

Description

@mrsibe

Context

The README is explicit: on first run "nothing is configured out of the box" — the user must open Settings and configure a model connection before the app can do anything. The app then either works or emits a failure the user has to interpret.

The failure modes are well known, and all distinguishable:

Symptom Actual cause
401 API key invalid, expired, or not sent
404 model id not available on that endpoint
ECONNREFUSED Ollama is not running
hang wrong base URL, or a proxy swallowing the request
garbage output the endpoint is OpenAI-compatible in name only

Today the user sees a raw error. AnythingLLM emphasises a no-frustrating-setup path; for a desktop app with bring-your-own-model this is the first five minutes of the product.

Proposal

A readiness panel that checks each dependency separately and says what to do:

Chat model        ✓ connected        (gpt-4o-mini @ api.openai.com)
Embedding model   ✓ ready            (multilingual-e5-small, 384d, local)
Vector store      ✓ ready
Test question     ✓ answered, 1.2s
  • Each row has its own Test action, so a failure is localised instead of reported as "chat is broken".
  • Errors are translated into an action ("Ollama is not running — start ollama serve and retry"), with the raw error available on demand.
  • The embedding row reports whether the model is downloaded and offers to download it (the app blocks remote loading and needs the model present).
  • The panel appears on first run and is reachable any time from Settings; a failing check blocks nothing else in the app.
  • A protocol mismatch (the endpoint answers, but not in the declared protocol's shape) is reported as such, not as a malformed answer.

Acceptance

  • First run with no configuration shows the readiness panel, not a chat box that fails later
  • Each of the five failure modes above produces a distinct, actionable message
  • Testing chat does not require typing a question into the real chat
  • A missing embedding model offers download, and the resulting vectors match the stored notebook width
  • No secret (API key) is ever written to logs

Verification

Manual against: a valid remote connection, an invalid key, an unknown model id, a stopped Ollama, and a bad base URL.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:uxInterface, workflow, information architectureenhancementNew feature or requestpriority:P1High value, current or next milestone

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions