Repository navigation
Conversation
Discovery skips transcript files last modified at or before modified_after := TIMESTAMP, so an incremental load reads only changed files (Claude, Claude Desktop, nested subagents, Codex rollouts). NULL or absent means no cutoff; columns are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
read_conversations()reads every transcript file on every call. On a working machine that is gigabytes of JSONL (here: ~800 Claude files and ~800 Codex rollouts, about 3 GB), so an incremental load that only wants the last few minutes still opens everything and filters afterwards. AWHERE timestamp > …cannot help: the files are read before the filter runs.Change
An optional named parameter on
read_conversations:During discovery, any transcript file whose filesystem modified time is at or before
modified_afteris skipped. This covers Claude and Claude Desktop transcripts, nestedsubagents/agent-*.jsonl, and Codexrollout-*.jsonl. Absent orNULLmeans no cutoff, and the columns are unchanged.A changed file is still read whole, since transcripts are appended to, so callers dedupe on
uuid. The cutoff is held for the duration of one call's load and restored afterwards, so other calls are unaffected.Tests
test/sql/modified_after.test: a cutoff before every fixture returns the same rows as no parameter (Claude with subagents, Codex). A cutoff after every fixture returns zero rows (Claude, Codex).NULLequals absent, a string literal is accepted, and the schema is unchanged.src/utils.rssets real file times on Claude, subagent and Codex files and checks that a 2010 cutoff keeps only the 2020 files, and that discovery sees every file again after the call.make release && make test_releaseon DuckDB 1.5.5: 22/22 SQL test files pass, andcargo test --libgives 7/7.🤖 Generated with Claude Code