Fingerprint runtime stats - #6085
Conversation
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Error ⚠.
Regression testing: 0/0 tests failed ⚠.
ec41695 to
5138a26
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Success ✓.
Regression testing: Success ✓.
The first 10 failing tests are:
noresetgen
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**
5138a26 to
054d984
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Success ✓.
Regression testing: Success ✓.
The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Error ⚠.
Regression testing: Success ✓.
The first 10 failing tests are:
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**
sql_logfill_autodisable [timeout]
e688cda to
4b4a9fb
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Success ✓.
Regression testing: Success ✓.
The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
truncatesc_offline_generated [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**
4b4a9fb to
2f70d0a
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Success ✓.
Regression testing: Success ✓.
The first 10 failing tests are:
sc_resume
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Success ✓.
Regression testing: Success ✓.
The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
noresetgen
reco-ddlk-sql **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
Adds per-fingerprint bufferpool page-in counters (total reads and the subset requiring actual disk I/O), tracked in a hash separate from the existing cost/time fingerprint tracking so a future write-side commit can attribute counters on a machine that never populated the main fingerprint hash for that query (e.g. a replication master applying a write schedule it never prepared itself). A thread-local pointer, set when a statement's fingerprint becomes known and cleared when the statement finishes, lets __memp_fget_internal() attribute each page fetch without threading SQL-layer state through berkdb. The mechanism is named generically (fingerprint_rtstats) since bufferpool page-ins are meant to be the first of several runtime metric categories attributed this way. Exposes the new counters as total_pagein_read/total_pagein_read_io columns on comdb2_fingerprints. Known limitation: Lua stored-procedure inner statements share the same fingerprint-set call but have no per-inner-statement clear point, so a schema/btree page read during one inner statement's prepare can be misattributed to the previous inner statement's fingerprint. Left for a follow-up commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mark Hannum <mhannum@bloomberg.net>
Extends the fingerprints.test suite with a case exercising the new total_pagein_read/total_pagein_read_io columns. Runs inserts and repeated selects against a dedicated table, then checks structural invariants (page-ins >= execution count, I/O count <= total count) rather than exact values, since raw page counts depend on bufferpool cache state and aren't deterministic across environments. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mark Hannum <mhannum@bloomberg.net>
Lua stored-procedure inner statements (db:exec/db:prepare, temp table handling) already set the page-in fingerprint thread-local correctly via the shared get_prepared_stmt_int() chokepoint, but had no per-inner-statement clear -- only the whole-procedure sqlite_done() call cleared it. This let page-ins between one inner statement finishing and the next one's own prepare bleed onto the previous statement's fingerprint instead of falling into NO-FINGERPRINT. Adds bdb_fingerprint_rtstats_clear() at every point an inner statement's execution conclusively ends: donate_stmt() (the shared terminal disposal point for exec/prepare/cursor handles -- covers dbstmt_free, dbstmt_exec's error path, dbstmt_fetch exhaustion, dbstmt_close, and reset_stmt's terminal branch) plus the six direct sqlite3_finalize() sites that bypass it (create_temp_table, dbtable_insert, dbtable_copyfrom, db_csvcopy, drop_temp_tables, clone_temp_tables). Deliberately does not hook lua_end_step(), since it fires multiple times against the same open statement in some paths (db_csvcopy, once per CSV row) and would clear prematurely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mark Hannum <mhannum@bloomberg.net>
Extends fingerprints.test with a stored procedure containing two distinct inner SQL statements, run multiple times, checking each inner statement's total_pagein_read/total_pagein_read_io invariants independently -- a scenario t10 didn't cover since it only exercised plain top-level SQL, not the lua inner-statement path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mark Hannum <mhannum@bloomberg.net>
2f70d0a to
4852f97
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Error ⚠.
Regression testing: Success ✓.
The first 10 failing tests are:
queuedb_rollover_noroll1_generated **quarantined**
noresetgen
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**
2 participants
.