Repository navigation
Bug: Persistent FTS index on an empty node table causes hash_index.cpp assertion after reopen #464
Description
Ladybug version
Rust crate: lbug = 0.16.1
What operating system are you using?
Ubuntu 24.04
What happened?
Creating a persistent FTS index on an empty node table succeeds, but after closing the process and reopening the
database from a new process, even a normal MATCH query fails with an internal assertion:
Assertion failed in file ".../lbug-src/src/storage/index/hash_index.cpp" on line 483:
hashIndexStorageInfo.overflowHeaderPage == INVALID_PAGE_IDX
This is different from #438: LOAD EXTENSION FTS succeeds when the Rust host binary is built with -rdynamic. The
failure happens after reopening a database that contains an FTS index created on an empty node table.
Are there known steps to reproduce?
Minimal Rust Reproduction
Cargo.toml:
[package]
name = "lbug_empty_fts_repro"
version = "0.1.0"
edition = "2021"
build = "build.rs"
[dependencies]
lbug = "0.16.1"build.rs:
fn main() {
// Needed for LadybugDB dynamic extensions to resolve symbols from the Rust host binary.
println!("cargo:rustc-link-arg=-rdynamic");
}src/main.rs:
use std::{env, process};
fn main() {
if let Err(err) = run() {
eprintln!("{err}");
process::exit(1);
}
}
fn run() -> Result<(), Box<dyn std::error::Error>> {
let mut args = env::args().skip(1);
let db_path = args.next().unwrap_or_else(|| "/tmp/lbug-empty-fts.db".to_string());
let mode = args.next().unwrap_or_else(|| "create-empty-index".to_string());
let db = lbug::Database::new(&db_path, lbug::SystemConfig::default())?;
let conn = lbug::Connection::new(&db)?;
match mode.as_str() {
"create-empty-index" => {
conn.query(
"CREATE NODE TABLE IF NOT EXISTS TestNode (
id STRING,
name STRING,
body STRING,
PRIMARY KEY(id)
);",
)?;
conn.query("INSTALL FTS;")?;
conn.query("LOAD EXTENSION FTS;")?;
conn.query(
"CALL CREATE_FTS_INDEX(
'TestNode',
'test_node_fts',
['name', 'body'],
stemmer := 'porter'
);",
)?;
conn.query("CHECKPOINT;")?;
println!("created FTS index on empty table");
}
"reopen-query" => {
conn.query("LOAD EXTENSION FTS;")?;
let result = conn.query("MATCH (n:TestNode) RETURN n.id LIMIT 5;")?;
println!("{result}");
}
"create-nonempty-index" => {
conn.query(
"CREATE NODE TABLE IF NOT EXISTS TestNode (
id STRING,
name STRING,
body STRING,
PRIMARY KEY(id)
);",
)?;
conn.query("CREATE (:TestNode {id: 'n1', name: 'hello', body: 'hello world'});")?;
conn.query("INSTALL FTS;")?;
conn.query("LOAD EXTENSION FTS;")?;
conn.query(
"CALL CREATE_FTS_INDEX(
'TestNode',
'test_node_fts',
['name', 'body'],
stemmer := 'porter'
);",
)?;
conn.query("CHECKPOINT;")?;
println!("created FTS index on non-empty table");
}
other => {
return Err(format!("unknown mode: {other}").into());
}
}
Ok(())
}Steps to Reproduce
rm -f /tmp/lbug-empty-fts.db
cargo run -- /tmp/lbug-empty-fts.db create-empty-index
cargo run -- /tmp/lbug-empty-fts.db reopen-queryActual Result
The second command fails with:
Assertion failed in file ".../lbug-src/src/storage/index/hash_index.cpp" on line 483:
hashIndexStorageInfo.overflowHeaderPage == INVALID_PAGE_IDX
Expected Result
Creating an FTS index on an empty table should either:
- be supported and reopen cleanly,
- fail gracefully when the index is created, or
- be ignored/deferred until rows exist.
It should not leave the database in a state that later triggers an internal assertion after reopen.
Control Case
If the table contains at least one row before creating the FTS index, reopening succeeds:
rm -f /tmp/lbug-empty-fts.db
cargo run -- /tmp/lbug-empty-fts.db create-nonempty-index
cargo run -- /tmp/lbug-empty-fts.db reopen-queryThis suggests the issue is specific to persistent FTS indexes created on empty node tables.
Relation to #438
This is related to Rust embedding and FTS, but it is not the same failure mode:
- Bug: FTS extension fails to load on macOS arm64 from Rust static binary (missing GDSFunction symbol) #438:
LOAD EXTENSION FTSfails because the host binary does not export LadybugDB symbols. - This issue:
LOAD EXTENSION FTSsucceeds with-rdynamic; the failure happens after creating a persistent FTS
index on an empty table and reopening the database.
Activity
adsharma commentedon May 7, 2026
I can't repro with:
Also ran your rust repro 3 times on ubuntu 24.04. Doesn't repro for me.
adsharma commentedon May 7, 2026
create_empty_fts.txt
reopen_query.txt
sify21 commentedon May 8, 2026
@adsharma I forgot to mention that I need to build lbug from source, or else it will report a lot of rust-lld error
.cargo/config.toml:In my environment it reproduces each time
My rust version is
cargo 1.94.1 (29ea6fb6a 2026-03-24)adsharma commentedon May 8, 2026
Yes - I built from source as well. Tested on both macos/arm64 and ubuntu 24.04/x64.
cargo 1.93.1 (083ac5135 2025-12-15)
Can you build a shell in your env (make shell in the ladybug source tree) and see if it repros?
If shell doesn't repro, but rust repros, it narrows the problems to rust bindings.
sify21 commentedon May 8, 2026
@adsharma Running
lbuginbuild/release/tools/shelldoesn't report error. The shell is built using tag v0.16.1adsharma commentedon May 18, 2026
Merged LadybugDB/extensions#1. Concurrent access to the unordered_map could be the root cause. Please retest when 0.17.x is released.
adsharma commentedon Jun 16, 2026
@sify21 did you get a chance to retest? Merged a couple of FTS related fixes into main.
kory-io commentedon Sep 28, 2026
I think I found the root cause. It also explains why this reproduces in some builds but not others.
Why only some builds fail. The failing check is a
DASSERT(hash_index.cpp:537on current main).DASSERTis only compiled in whenRUNTIME_CHECKSis defined orNDEBUGis not (common/assert.h), and a failure throws anInternalException.make release+ the shell does not reproduce.build.rsenables runtime checks for its debug and relwithdebinfo builds, which matches the from-sourcecargobuild reproducing every time.In release builds the state below is still written. It just goes unchecked.
Root cause:
PrimaryKeyIndex::checkpoint(hash_index.cpp:765-780) sets the two header pointers under different conditions.firstHeaderPageis only assigned inwriteHeaders, which runs underif (indexChanged).updateOverflowHeaderPageIfNeededruns whenever an overflow file exists (a STRING primary key), regardless ofindexChanged, and setsoverflowHeaderPageif it is unset.When a checkpoint processes the index of a node table with zero rows,
indexChangedis false. So it persistsfirstHeaderPage == INVALID_PAGE_IDXtogether with a validoverflowHeaderPage. On the next open, the constructor takes thefirstHeaderPage == INVALID_PAGE_IDXbranch and assertsoverflowHeaderPage == INVALID_PAGE_IDX, which is the combination the checkpoint just wrote.Minimal repro without FTS, run with a shell built from main (8caf03b) with
-DENABLE_RUNTIME_CHECKS=TRUE:Reopening the database, even for
MATCH (n:t) RETURN count(*);, fails withAssertion failed ... hash_index.cpp on line 537: hashIndexStorageInfo.overflowHeaderPage == INVALID_PAGE_IDX.Controls on the same build, all of which reopen cleanly:
ALTER;INT64primary key (no overflow file);CHECKPOINTand noALTER(that checkpoint does not reach this index).I haven't traced the FTS path. Presumably creating the FTS index on the empty table, like the
ALTER, is what makes the checkpoint write this empty index.Possible fixes:
Happy to open a PR for whichever you prefer.
adsharma commentedon Sep 28, 2026
Please send a pr with: assign both pointers under the same condition
fix(storage): assign PK index header pages under one condition