fix(cluster): reject non-finite and zero-norm embeddings in Scorer.Route

[sanidhyasin]

Fixes #780.

Problem

After a successful Embed, Scorer.Route checked only the embedding
dimension. A dimensionally valid zero, NaN or Inf vector went
straight into topPNearest and produced a routing decision.

Neither case has meaningful cosine ordering:

• Zero vector — every centroid similarity ties at zero, so
  sort.SliceStable's index tie-break decides the top-p. Centroid order
  in the artifact becomes the routing feature.
• NaN/Inf — the comparator no longer defines a strict weak ordering,
  so the selected centroids aren't semantically valid. Go doesn't panic
  here; routing just succeeds with a wrong answer.

Either way the request is served by a model chosen on artifact layout
rather than on the prompt, and nothing upstream reports it. That's the
fail-open shape the package rules out: every other failure mode in
Route (embed timeout, embed error, dim mismatch, alpha-vector length,
empty argmax) returns ErrClusterUnavailable → 503.

The embedder can't be relied on to catch this. hugot's WithNormalization
floors the denominator at 1e-12, so a zero vector stays zero and NaNs
stay NaN — the ONNX path can return a numerically corrupted vector with
no error, and onnxEmbedder.Embed validates only count and dim.

Production incidence is unproven — I have no telemetry or ONNX
reproduction, only the code-level gap. Treating this as a correctness
fix, not an active incident.

Changes

internal/router/cluster/scorer.go:

• New validateEmbedding(vec, wantDim) (reason, error), called at the
  dim check it replaces. Scorer.Route is the only place a
  runtime-produced vector enters the package (distribution.go's
  topPNearest call feeds it centroid rows, which are trusted artifact
  data), so validating there covers the surface — and it sits on
  Scorer rather than in the ONNX adapter, since Scorer can be handed
  any Embedder implementation.
• Failures map to ErrClusterUnavailable. The dim-mismatch error string
  is unchanged, so its 503 mapping and message are byte-identical.
• The rejection log carries reason — one of wrong_dimension, nan,
  positive_inf, negative_inf, zero_norm — plus embedder, both
  dims and requested_model. That's the countable signal the issue asks
  for, in the package's existing structured-log idiom rather than a new
  metrics dependency. No vector values or prompt text are logged. The
  dim-mismatch log message becomes the shared "invalid embedding" line;
  reason=wrong_dimension keeps that case distinguishable.

The norm test is exact zero, not an epsilon. A tiny but finite vector
still orders centroids meaningfully, so rejecting it would turn working
requests into 503s; zero and non-finite categorically don't. No
norm ≈ 1 check either — that needs real Jina/Qwen output norms first.

Package CLAUDE.md/AGENTS.md fail-open list updated (mirrored, per the
mirror notice).

Tests

TestScorer_ReturnsErrOnInvalidEmbedding — table-driven over all-zero,
NaN at beginning/middle/end, +Inf and -Inf. Each asserts Route returns
ErrClusterUnavailable and that the rejection is categorized correctly.
Every case routes successfully on main.

TestScorer_RoutesTinyNonZeroEmbedding — a 1e-20 vector must still
route, pinning the exact-zero boundary so a later epsilon can't creep in
unnoticed.

TestScorer_ReturnsErrOnDimMismatch is unchanged and still passes
through the new helper.

Validation

go test -tags no_onnx ./internal/router/... — all packages pass.
go build -tags no_onnx ./... and gofmt -l clean.

I don't have the ONNX assets or libonnxruntime locally, so the
onnx_integration parity test wasn't run; this change doesn't touch the
embedder implementations.

Not in this PR

Per the issue's scope note, the AlphaFloor / dial-calibration work
(#779) is left alone.

[greptile-apps]

PR author is not in the allowed authors list.

[devin-ai-integration]

Thanks a lot for this — really nice contribution, and it lands squarely inside the repo's conventions, so no changes needed from our side.

What I checked and liked:

• Right layer. validateEmbedding sits on Scorer in internal/router/cluster, the single point a runtime-produced vector enters the package, rather than in the ONNX adapter — so it holds for any Embedder implementation. No new imports or dependencies (math/errors only), so the layer model and import rules in root AGENTS.md are untouched.
• Fail closed via the sentinel. Every rejection wraps ErrClusterUnavailable, matching the package rule "Don't add fail-open fallbacks" — and you kept the dim-mismatch error string byte-identical, so the existing 503 mapping and TestScorer_ReturnsErrOnDimMismatch still hold. Updating the fail-open list in both internal/router/cluster/AGENTS.md and CLAUDE.md per the mirror notice is exactly right.
• Structured logging. Categorized reason constants (wrong_dimension/nan/positive_inf/negative_inf/zero_norm) instead of ad-hoc strings, emitted through the package's existing slog idiom with no vector values or prompt text — the countable signal the issue asked for without a new metrics dependency.
• Comments. Why-focused only (the hugot 1e-12 denominator floor, and why the zero test is exact rather than an epsilon) — hidden constraints that aren't inferable from the code, which is the bar in root AGENTS.md ("Concise comments, sparingly… only when why is non-obvious"). No PR/issue refs or pre-fix narrative in the code itself; the context lives in the PR description where it belongs.
• Non-tautological tests. Table-driven over zero/NaN-at-three-positions/±Inf, each asserting errors.Is(err, ErrClusterUnavailable) from Route plus the reason category, and TestScorer_RoutesTinyNonZeroEmbedding pins the 1e-20 boundary so a future epsilon can't silently turn working requests into 503s. All would fail if the production gate were deleted, and the fixtures follow this file's existing makeOpusVec / literal-model-ID pattern.

Also appreciated the honesty about production incidence being unproven and the explicit scope note leaving the AlphaFloor work alone. Verified locally: make precommit is clean (fmt + vet + build + full test suite), and CI is green.