local-ai-use: setup script makes five environment assumptions that fail silently
Summary
Setup script makes five environment assumptions that may fail silently.
Issues
1. CLI selected by name, not capability. find_cli() searches PATH for lemonade before lemonade-server and uses whichever it finds first. The lemonade-sdk pip package installs both: lemonade (an eval/benchmarking tool with no serve subcommand) and lemonade-server (the REST server). Pip users get the wrong binary, lemonade serve exits without error and without a server, and setup burns 90 s on a health check that can never pass. Fix: probe the candidate's --help for serve/pull subcommands instead of trusting the name.
2. Port 13305 hardcoded as the definition of "reachable." The script only calls /health on DEFAULT_PORT, even though the CLI ships a status subcommand that reports the real bound host/port. Any server on a non-default port (existing config, different install channel, port conflict) reads as "not running," so the script launches a redundant instance. Fix: discover the endpoint via <cli> status, fall back to the default only if unavailable.
3. Wrong env var for the download path. The script sets LEMONADE_CACHE_DIR, but the actual multi-GB download lands wherever HF_HOME/HUGGINGFACE_HUB_CACHE points (a Hugging Face Hub convention the router uses internally). These are independent knobs. A user with HF_HOME already set — common on shared ML machines — gets a silent mismatch: setup reports success, then the first download goes somewhere never validated. Fix: set both consistently, or query the server for its effective cache path.
4. No preflight write/space check before a multi-GB transfer. The script and the rule's failure-handling section jump straight to pull <model> with no check that the target is writable or has room. Quota, permissions, or a read-only mount only surface minutes later, buried in low-level retries (10 attempts, exponential backoff), instead of failing fast.
5. No bounded failure signal for a stuck pull. The rule says "if the model is missing, run pull and retry once" but never distinguishes a slow pull from a broken one — a healthy 5 GB CPU-bound pull and one that resets every ~1.5 GB both just print Progress: NN%. Fix: cap the wait, and treat more than one reset to 0% as failure with a pointer to the server log.
Feedback kindly provided by Nabeel Shirazi
local-ai-use: setup script makes five environment assumptions that fail silentlySummary
Setup script makes five environment assumptions that may fail silently.
Issues
1. CLI selected by name, not capability.
find_cli()searchesPATHforlemonadebeforelemonade-serverand uses whichever it finds first. Thelemonade-sdkpip package installs both:lemonade(an eval/benchmarking tool with noservesubcommand) andlemonade-server(the REST server). Pip users get the wrong binary,lemonade serveexits without error and without a server, and setup burns 90 s on a health check that can never pass. Fix: probe the candidate's--helpforserve/pullsubcommands instead of trusting the name.2. Port 13305 hardcoded as the definition of "reachable." The script only calls
/healthonDEFAULT_PORT, even though the CLI ships astatussubcommand that reports the real bound host/port. Any server on a non-default port (existing config, different install channel, port conflict) reads as "not running," so the script launches a redundant instance. Fix: discover the endpoint via<cli> status, fall back to the default only if unavailable.3. Wrong env var for the download path. The script sets
LEMONADE_CACHE_DIR, but the actual multi-GB download lands whereverHF_HOME/HUGGINGFACE_HUB_CACHEpoints (a Hugging Face Hub convention the router uses internally). These are independent knobs. A user withHF_HOMEalready set — common on shared ML machines — gets a silent mismatch: setup reports success, then the first download goes somewhere never validated. Fix: set both consistently, or query the server for its effective cache path.4. No preflight write/space check before a multi-GB transfer. The script and the rule's failure-handling section jump straight to
pull <model>with no check that the target is writable or has room. Quota, permissions, or a read-only mount only surface minutes later, buried in low-level retries (10 attempts, exponential backoff), instead of failing fast.5. No bounded failure signal for a stuck pull. The rule says "if the model is missing, run pull and retry once" but never distinguishes a slow pull from a broken one — a healthy 5 GB CPU-bound pull and one that resets every ~1.5 GB both just print
Progress: NN%. Fix: cap the wait, and treat more than one reset to 0% as failure with a pointer to the server log.Feedback kindly provided by Nabeel Shirazi