Repository navigation
feat(sessions): add structured local history search #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,7 @@ use std::time::Duration; | |
| use rusqlite::{Connection, OptionalExtension, params}; | ||
| use xai_sqlite_journal::JournalMode; | ||
|
|
||
| use crate::query::SessionQuery; | ||
| use crate::recovery; | ||
|
|
||
| /// Bump when making breaking schema changes that require dropping and recreating tables, or to force a rebuild of stale index content. | ||
|
|
@@ -439,7 +440,8 @@ impl SessionSearchIndex { | |
| Ok(ids) | ||
| } | ||
|
|
||
| /// When that intersection matches nothing the query reruns as an OR, so partial matches still show up. A query shaped | ||
| /// Plain keywords use AND first, then OR when there are no hits. Explicit | ||
| /// phrases, field scopes, and exclusions use a strict plan with no fallback. A query shaped | ||
| /// like a session id (a full UUID or a hyphenated hex prefix) matches `session_docs.session_id` directly. FTS only | ||
| /// indexes title and content, and a hyphenated UUID `MATCH` looks for tokens that were never indexed. | ||
| pub fn query( | ||
|
|
@@ -457,18 +459,21 @@ impl SessionSearchIndex { | |
| } | ||
| } | ||
|
|
||
| let Some((and_query, or_query)) = Self::build_match_queries(query) else { | ||
| let plan = SessionQuery::parse(query); | ||
| let Some(primary) = plan.primary_match() else { | ||
|
Comment on lines
+462
to
+463
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
A negative-only hexadecimal query can return precisely the session it appears to exclude. For example, Useful? React with 👍 / 👎. |
||
| return Ok(QueryResult { | ||
| results: Vec::new(), | ||
| next_offset: None, | ||
| total_estimate: Some(0), | ||
| }); | ||
| }; | ||
|
|
||
| let result = self.run_match_query(&and_query, cwd, limit, offset, include_content)?; | ||
| let result = self.run_match_query(primary, cwd, limit, offset, include_content)?; | ||
| // Gate the fallback on the total (not the page) so every offset of one logical query is served by the same match string | ||
| if result.total_estimate == Some(0) && and_query != or_query { | ||
| return self.run_match_query(&or_query, cwd, limit, offset, include_content); | ||
| if result.total_estimate == Some(0) | ||
| && let Some(fallback) = plan.fallback_match() | ||
| { | ||
| return self.run_match_query(fallback, cwd, limit, offset, include_content); | ||
| } | ||
| Ok(result) | ||
| } | ||
|
|
@@ -623,64 +628,6 @@ impl SessionSearchIndex { | |
| total_estimate: Some(total_usize), | ||
| }) | ||
| } | ||
|
|
||
| /// Build the AND-joined and OR-joined FTS5 MATCH strings for a query. | ||
| /// | ||
| /// The strings are identical for single-token queries, which lets the caller skip the fallback rerun. | ||
| fn build_match_queries(query: &str) -> Option<(String, String)> { | ||
| let prefixes: Vec<String> = query | ||
| .split_whitespace() | ||
| .flat_map(Self::sanitize_token) | ||
| .map(Self::token_prefix) | ||
| .collect(); | ||
|
|
||
| if prefixes.is_empty() { | ||
| let fallback = query.trim(); | ||
| if fallback.is_empty() { | ||
| return None; | ||
| } | ||
| let cleaned = fallback.replace('"', ""); | ||
| let phrase = format!("\"{cleaned}\" *"); | ||
| return Some((phrase.clone(), phrase)); | ||
| } | ||
|
|
||
| Some((prefixes.join(" AND "), prefixes.join(" OR "))) | ||
| } | ||
|
|
||
| /// Split a query word on every stripped character instead of gluing: `session_picker.rs` searches as `session_picker` and `rs`. | ||
| /// Gluing produced the never-indexed `session_pickerrs`. | ||
| /// Fragments with no alphanumeric (`-`, `->`, `_`) are dropped: they tokenize to empty phrases; an empty phrase inside an AND matches nothing. | ||
| fn sanitize_token(token: &str) -> impl Iterator<Item = &str> { | ||
| token | ||
| .split(|c: char| !(c.is_ascii_alphanumeric() || c == '_' || c == '-')) | ||
| .filter(|part| part.chars().any(|c| c.is_ascii_alphanumeric())) | ||
| } | ||
|
|
||
| /// One quoted FTS5 prefix per token, stemmed on the query side only. Plural queries reach singular docs by searching the | ||
| /// shorter stem (`sessions` becomes `session*`, `caches` becomes `cach*`). Words shorter than four letters, identifiers | ||
| /// with digits/`_`/`-`, and words ending in `ss` (`pass`, `class`) stay exact. | ||
| fn token_prefix(token: &str) -> String { | ||
| let stem = if token.len() < 4 || !token.chars().all(|c| c.is_ascii_alphabetic()) { | ||
| token | ||
| } else { | ||
| let lower = token.to_ascii_lowercase(); | ||
| if lower.ends_with("es") { | ||
| // Stemming `caches` to `cach` still finds `cache`: the trailing `*` covers the dropped `e` | ||
| match token.len().checked_sub(2).and_then(|n| token.get(..n)) { | ||
| Some(s) => s, | ||
| None => token, | ||
| } | ||
| } else if lower.ends_with('s') && !lower.ends_with("ss") { | ||
| match token.len().checked_sub(1).and_then(|n| token.get(..n)) { | ||
| Some(s) => s, | ||
| None => token, | ||
| } | ||
| } else { | ||
| token | ||
| } | ||
| }; | ||
| format!("\"{stem}\" *") | ||
| } | ||
| } | ||
|
|
||
| /// True when `query` is a UUID or a hyphenated hex prefix long enough to be an intentional session-id search (not a normal keyword). | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the picker is on
SourceFilter::All, this early return suppresses foreign entries as well as native ones becauseis_content_search_disabled()is true only forExternal. For example, a Codex session titledtitle:rust notespreviously matchedtitle:ruston the All page but now disappears. Foreign histories are not FTS-indexed, so deep-search results cannot restore it. Apply structured-query suppression per entry to native sources while retaining literal matching for foreign entries.Useful? React with 👍 / 👎.