Skip to content

tackle-todos: projection pushdown, cast-free equality, durable codec, TUI fetch/cancel, docker .resource fix - #65

Open
moriyoshi wants to merge 6 commits into
mainfrom
chore/tackle-todos-2026-10
Open

moriyoshi wants to merge 6 commits into
mainfrom
chore/tackle-todos-2026-10

Conversation

@moriyoshi

Copy link
Copy Markdown
Owner

A tackle-todos sweep that closes ten open TODO.md items in five independent units, one commit each, plus a docs commit.

Changes

  • TUI (imbh-tui):
    • The waterfall is fetched once the cursor settles (120 ms debounce, Enter skips it), and the last 16 traces are kept in an LRU.
    • The waterfall pane has its own spinner.
    • Ctrl-C cancels a query in any mode and never quits; Esc cancels while a query holds the keyboard.
    • Trace ids are kept on the snapshot instead of being re-parsed from the row text.
  • Projection pushdown (imbh-query): each segment's Parquet reader gets a ProjectionMask matched by column name, and the buffer is coerced to the same projected schema.
    • Guarded by an on/off batch-for-batch equality test over 35 queries on a schema-evolved corpus.
    • Measured: logs count(*) 8.1 → 1.7 ms; spans hex(trace_id) = … 7.1 → 2.3 ms; matches(name, …) 10.8 → 4.0 ms at mid cardinality.
  • Equality filters (imbh): attr_eq_sql compares a promoted key without CAST, about 0.1–0.5 ms faster per 100k rows, with unchanged results.
    • The PromQL-agreement test moves into imbh-lgtm and now calls the real collapse_duplicate_samples.
  • Compression codec (imbh-storage / imbh): the codec is durable in db.info, like promote and retention, so the housekeeper and compaction write with the host's codec.
    • A capped pass takes partitions oldest-first, ordered by day, then earliest segment time, then table order.
  • Docker (imbh-server): a remap script's .resource now refreshes after network discovery.
    • The old seed also stripped container.network.* from records whose script set .resource.
    • New opt-in test of the plugin against a real daemon.

Behaviour changes worth a look

  • Db::compact_bounded spends its budget oldest-first. Before, it worked through every logs partition before any spans partition.
  • ScanStats::bytes_scanned counts only projected columns, so it reads about 0 for count(*).
  • App::apply (TUI) releases the input lock only for a current-generation result.
  • Every fresh snapshot clears the TUI waterfall LRU, auto-refresh included.
  • The public API changes are additive only: Db::compression() and read_compression. A database written by 0.9.0 opens unchanged.

Not covered

  • Nobody exercised the TUI interactively; it is covered by unit tests and TestBackend renders.
  • The daemon test has not been run against a real daemon. To run it: IMBH_DOCKER_DAEMON_E2E=1 cargo test -p imbh-server --test docker_plugin_daemon -- --ignored --nocapture.
  • Deferred to a 0.10 API pass: a traces().get() lower bound, time-bounding the metrics dimensions/series/exemplars APIs, and #[non_exhaustive] on DbStats.

Verification

cargo fmt --all --check, cargo build --workspace, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass: 779 passed, 0 failed, 5 ignored.

🤖 Generated with Claude Code

moriyoshi and others added 6 commits October 19, 2026 12:49
…n-flight queries

Keep trace ids on the snapshot instead of re-parsing row text; fetch a
trace's waterfall only once the cursor settles (120 ms, Enter skips) and
serve revisits from a 16-entry LRU. The waterfall pane draws its own
spinner without the banner or input lock. Ctrl-C (any mode) and Esc
(while a query holds the keyboard) cancel by generation; App::apply now
releases the lock only for a current-generation result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build a by-name ProjectionMask per segment and coerce the buffer snapshot
to the same projected schema, so narrow queries no longer decode the wide
JSON columns. Guarded by an on/off batch-for-batch equality test over a
schema-evolved corpus. bytes_scanned now counts projected columns only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dedup against the real function

attr_eq_sql compares a promoted dictionary column without CAST, falling
back to the JSON only where the column is NULL. The PromQL-agreement test
moves into imbh-lgtm so it calls collapse_duplicate_samples directly
instead of a transcribed copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…partitions first

The codec becomes durable state like promote and retention: recorded in
db.info, inherited when DbBuilder::compression is omitted, and used by the
housekeeper and in-process compaction. A capped pass (prepare_pending
max_jobs, Db::compact_bounded) now takes partitions oldest-first with a
deterministic tiebreak.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry; opt-in daemon test

The remap seed was frozen at StartLogging, so scripts saw a stale
.resource and an override built from it dropped container.network.*
attributes. Add an opt-in (#[ignore] + IMBH_DOCKER_DAEMON_E2E=1) test of
the plugin packaging path against a real daemon; build.sh honours
CARGO_TARGET_DIR and an optional PLUGIN_STAMP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…OURNAL)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant