Skip to content

feat: identify slow reads in bounded recovery signals - #392

Merged
beinan merged 1 commit into
mainfrom
codex/slow-read-recovery-signals
Oct 19, 2026
Merged

beinan merged 1 commit into
mainfrom
codex/slow-read-recovery-signals

Conversation

@beinan

@beinan beinan commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Production recorded /records reads taking 217–241 seconds before returning 404. The recovery endpoint retained only 5xx outcomes, while route-level latency histograms could not identify the affected table.

Retain the latest completed read taking at least five seconds in the existing bounded per-table hint cache, including its status and response-header duration. These additive schema-1 fields preserve the nonblocking recording path, five-minute expiry, 4,096-entry cap and 1 MiB response limit. Failures and slow reads precede ordinary activity when the response is truncated. Request parameters, record IDs and payloads are excluded. This supplies diagnostic hints; it does not cancel requests or infer that a slow read needs a merge.

Validation: metrics unit tests passed (10, plus one ignored manual benchmark); enabled/disabled HTTP integration tests passed, including a delayed 404, in-flight demand and payload exclusion. Tests also cover threshold boundaries, independent expiry, preservation after successful requests, lock contention and bounded truncation. cargo clippy -p lance-context-metrics --all-targets -- -D warnings, formatting and diff checks passed. This is independent of #390 and does not block its release.

@beinan
beinan merged commit efbde80 into main Oct 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant