[b3hr4d]

Summary

I ran the on-device AI with the pinned model in the cloud, so Behrad could try it without using his own connection. It works. B3Note's own localAi.ts downloaded the model (869 MB) from the pinned commit, web-llm's hash checks passed, and Ask and Summarize both answered correctly.

The same run found one bug, which this PR fixes. On a GPU without 16-bit float support, the app offered the 1 GB download, downloaded all of it, and then failed.

The bug

• web-llm 0.2.85's record for Qwen2.5-1.5B-Instruct-q4f16_1-MLC has no required_features, though its library's shaders use 16-bit floats.
• So on a GPU without WebGPU's shader-f16, web-llm downloads the config, the library, the tokenizer and all 30 weight shards first. Then it fails to build its shaders (GPUPipelineError; WGSL says "extension 'f16' is not allowed").
• detectLocalAi only checked that a GPU exists, so the app offered "About 1 GB, once".
• After the download, the app showed "The model stopped before it finished. Try again." Trying again fails the same way.
• Fact (run here): in Chromium 153, whose software GPU lacks shader-f16, with the production CSP, the app offered the download. The failed load left 881 MB in the browser's storage.

Changes

• frontend/src/lib/localAi.ts:
  • detectLocalAi asks for the GPU web-llm uses (powerPreference: "high-performance"). If that GPU lacks shader-f16, the status is unavailable with a new reason, NO_F16_REASON, before any download.
  • The pinned record adds required_features: ["shader-f16"]. If web-llm ever gets such a GPU, it now stops before the weights.
  • localAiError shows NO_F16_REASON for web-llm's ShaderF16SupportError.
  • localAiError also returns both unavailable reasons as they are. They name a GPU, so they used to match "ran out of memory".
  • prepareLocalAi throws the reason detection found.
• README.md: the open model needs a GPU with 16-bit float support.

Verification

• Fact (run here): in frontend/, pnpm format:check, pnpm typecheck, pnpm test (9 files, 60 tests) and pnpm build pass.
• Fact (run here): six injected faults each failed a test, and each was reverted:
  • no shader-f16 check;
  • the default GPU instead of web-llm's;
  • no required_features;
  • web-llm's f16 error read as running out of memory;
  • prepareLocalAi throwing the generic reason;
  • the reasons read as running out of memory.
• Fact (run here): the same Chromium page, with this change:
  • detection reports the model unavailable, with the new reason;
  • asked to run anyway, the app shows that reason;
  • web-llm called directly with the pinned record stops with ShaderF16SupportError with 5 MB stored, instead of 881 MB;
  • there are no CSP violations.
• Fact (run here): with a GPU that has shader-f16, detection still offers the download, and the model still loads with the new record. That GPU was Mesa's software Vulkan driver, through Dawn (Chrome's WebGPU engine), in Node, running localAi.ts itself.
• Fact (run here, before this change, same setup):
  • it downloaded the 30 shards from the pinned commit (869 MB, through us.aws.cdn.hf.co), and web-llm accepted all three hashes;
  • Ask ("When is my dentist appointment, and what do I need to bring?") and Summarize answered correctly, in about 2 minutes each on that CPU-only GPU.
• Open: not run on a real GPU.

Notes

• In Chrome, when Chrome's built-in model is available, the app uses Gemini Nano instead, and Chrome downloads it itself. This PR doesn't change that.

---

Generated by Claude Code