perf(lava): WAL journaling for the LAVA database, gated on local storage

[abrignoni]

Summary

Enables SQLite WAL journaling (+ synchronous=NORMAL) for _lava_artifacts.db when — and only when — the output path is confirmed local storage. The media insert helpers commit per row, so media-heavy runs pay thousands of fsync'd rollback-journal commits; in a local benchmark this change moves per-row commit throughput from ~2,100/s to ~68,500/s.

Supersedes #285 — original work by @OneSixForensics. This PR keeps his design (safe-by-default network detection, WAL take-effect verification per @JamesHabben) and adds three fixes found during review:

Fixes on top of #285

1. Finalize-time checkpoint + journal reset (data-completeness fix). journal_mode=WAL is persisted in the database header. Functional testing showed the finalized db from #285, copied to read-only/write-blocked media, fails to open with attempt to write a readonly database — SQLite must create a -shm file to read a WAL database, and the LAVA viewer opens the db with OPEN_READONLY. lava_finalize_output() now runs PRAGMA wal_checkpoint(TRUNCATE) + PRAGMA journal_mode=DELETE before close, so the delivered .db is complete, standalone (no -wal/-shm sidecars), back on the universal delete journal, and openable from archived or write-blocked report media.

2. macOS local-disk detection. #285 detected local storage on Windows (GetDriveTypeW) and Linux (/proc/mounts) but returned "undetermined" on macOS, so WAL never engaged there. Added detection via mount output using the local option (the MNT_LOCAL flag); SMB/NFS/AFP volumes and unrecognized mounts stay on the safe rollback journal.

3. Narrowed exception handling so the PR introduces no new pylint warnings (--disable=C,R clean except pre-existing lavafuncs debt).

Forensic safety notes

• storage_safety.py only ever touches the LAVA report database connection (report output, never evidence); it opens no files in write mode itself.
• Network/undetermined paths keep SQLite's default rollback journal — WAL is never enabled on NAS/UNC/mapped-drive output, per SQLite's documented WAL limitation.
• Windows extended-length paths are handled: \\?\C:\... is local, \\?\UNC\... is network (RLEAPP prefixes \\?\ to all output paths on Windows).

Validation

• End-to-end test driving initialize_lava → per-row commits → lava_finalize_output: WAL active during the run, sidecars removed at finalize, header back to delete mode, a .db-only copy contains all rows, PRAGMA integrity_check ok, and read-only open from a read-only directory succeeds.
• Detection unit-tested with injected mount tables (macOS smb/apfs/autofs, Linux nfs4/ext4, UNC and \\?\ extended paths).
• pylint --disable=C,R: no new warnings vs main.
• PluginLoader smoke test: 316 plugins on this branch, identical to main.

Supersedes #285 — original work by @OneSixForensics.

🤖 Generated with Claude Code

[abrignoni]

@JamesHabben @Johann-PLW — requesting your review before this merges, since it touches the LAVA framework (lavafuncs.py) and the delivered report DB.

Two areas worth the most scrutiny:

1. lava_finalize_output() → finalize_lava_journal_mode (the forensic-completeness part). During a run the DB is in WAL, so data lives in -wal/-shm sidecars; at finalize we PRAGMA wal_checkpoint(TRUNCATE) + PRAGMA journal_mode=DELETE so the delivered .db is a single standalone file with no sidecars, and opens read-only from write-blocked/archived media. Without this, copying the .db alone could silently drop uncheckpointed rows, or fail to open read-only (attempt to write a readonly database). I verified end-to-end: 500 per-row commits under WAL, finalize back to delete mode, sidecars gone, .db-only copy in a read-only dir opens with all rows and integrity_check=ok.

2. Local-vs-network detection in storage_safety.py. WAL only engages on confirmed local storage; SMB/NFS/AFP/UNC/mapped drives and any undetermined path stay on the default rollback journal (SQLite's documented WAL-over-network limitation). Detection covers Windows (GetDriveTypeW, incl. \\?\ extended paths), Linux (/proc/mounts), and macOS (mount local / MNT_LOCAL). Please sanity-check the platform edge cases against setups you run.

No rush — holding the merge until one of you has looked. Supersedes #285 (original work by @OneSixForensics).

[abrignoni]

Superseded in part by RLEAPP #402, which takes only the finalize-time checkpoint from this PR and applies it to all five cores.

The WAL enablement and the storage_safety.py module are deliberately not carried over. That is about timing rather than merit: lavafuncs.py is meant to stay identical across the extractors and has already drifted five ways, a consolidation of the cores is planned, and landing a new ~300-line core module in one repo now would deepen that split. The performance case still stands and is better applied once, against the unified core.

Keeping this open so the WAL work is not lost. Original work in #285 by @OneSixForensics.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

[abrignoni]

Correction to my previous comment. I split the finalize checkpoint out of this PR into five separate PRs and have now closed all of them, because the split does not hold together.

WAL is enabled in ilapfuncs.py on _Timeline/tl.db and _KML Exports/_latlong.db, never on _lava_artifacts.db — the only database lava_finalize_output touches. Without the enablement from this PR, the checkpoint guards a state that database cannot reach, so it was dead code. The checkpoint and the WAL enablement are one unit and should land together.

This PR stays open on its own merits. The performance case is unaffected by any of the above.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com