Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions crates/codegen/xai-grok-pager/docs/user-guide/17-sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,51 @@ grok sessions search "rate limit"

`grok sessions list` shows sessions for the current working directory, grouped by worktree label. Each row lists the session ID, the creation and update dates, the source status, and the summary. `grok sessions search` combines a local SQLite index with remote results.

### Structured local search

Use quotes, field scopes, and exclusions to narrow the local session history:

```bash
# An exact sequence of words in either the title or prompts
grok sessions search '"rate limit"'

# Combine a title prefix, an exact prompt phrase, and a title exclusion
grok sessions search 'title:retry content:"rate limit" -title:archived'

# Exclude a word or phrase from both fields
grok sessions search 'rust -tokio -"out of date"'

# Ordinary keyword search without remote results
grok sessions search --local 'borrow checker'
```

| Syntax | Meaning |
|---|---|
| `rate limit` | Match both word prefixes; if there are no matches, offer sessions matching either word |
| `"rate limit"` | Match adjacent words in order, without prefix expansion or plural stemming |
| `title:retry` | Match a word prefix in the session title |
| `content:"rate limit"` | Match a phrase in the indexed prompts |
| `-archived` | Exclude sessions matching this prefix in either field |
| `-title:archived` | Exclude sessions matching this prefix in the title |

Explicit syntax is strict: all positive clauses must match and every exclusion
must be absent. These queries never fall back to partial matches. Quotes match
SQLite's indexed words, so case and punctuation are normalized. An unfinished
quote extends to the end of the query while you type. Empty clauses are ignored;
a query containing only exclusions returns no results. Field names are lowercase
and must touch their value. Unknown prefixes and words such as `OR` are searched
as ordinary text, not interpreted as operators.

Structured CLI queries automatically search the local index only and print a
notice. The remote keyword service does not implement this syntax. `--local`
also forces local-only behavior for ordinary keywords. If local session search
is disabled, both forms report an error instead of returning unconstrained
remote results. Local results remain scoped to the current working directory.

The same syntax is available in the native session picker's deep content search
and the `x.ai/session/search` ACP extension's `query` field. Foreign-agent
histories and remote chat search retain their existing search behavior.

---

## The grok usage Subcommand
Expand Down
50 changes: 41 additions & 9 deletions crates/codegen/xai-grok-pager/src/sessions_cmd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ use clap::Subcommand;
use xai_grok_login::{AuthManager, try_ensure_fresh_auth};
use xai_grok_shell::agent::config::Config as AgentConfig;
use xai_grok_shell::session::merge::MergedSession;
use xai_grok_shell::session::storage::search::SessionQuery;
use xai_grok_shell::util::grok_home::grok_home;
#[derive(Debug, clap::Args, Clone)]
pub struct SessionsArgs {
Expand All @@ -18,10 +19,14 @@ enum SessionsCommand {
#[arg(short = 'n', long, default_value = "20")]
limit: usize,
},
/// Search sessions by keyword
/// Search session titles and prompts with keywords, phrases, or field filters
Search {
/// Search query (searches summaries and first prompts).
/// Query: words, "exact phrases", title:term, content:term, or -term.
/// Structured queries search the local index only.
query: String,
/// Search only the local index, without remote keyword results
#[arg(long)]
local: bool,
/// Maximum number of sessions to show
#[arg(short = 'n', long, default_value = "20")]
limit: usize,
Expand All @@ -34,14 +39,24 @@ enum SessionsCommand {
}

pub async fn run(args: SessionsArgs, agent_config: &AgentConfig) -> Result<()> {
let local_only = matches!(
&args.command,
SessionsCommand::Search { query, local, .. }
if *local || SessionQuery::parse(query).is_structured()
);
// Best-effort only: never force an interactive public login here. Enterprise deployments may configure only a
// deployment_key and a custom xai_api_base_url. Otherwise we still proceed so the SessionRegistryClient can use
// the deployment_key when talking to the custom proxy.
let auth = try_ensure_fresh_auth(
&agent_config.grok_com_config,
agent_config.endpoints.proxy_url(),
)
.await;
// A local search must not wait for an authentication refresh either.
let auth = if local_only {
None
} else {
try_ensure_fresh_auth(
&agent_config.grok_com_config,
agent_config.endpoints.proxy_url(),
)
.await
};

let auth_manager = std::sync::Arc::new(AuthManager::new_with_proxy_base_url(
&grok_home(),
Expand Down Expand Up @@ -73,7 +88,11 @@ pub async fn run(args: SessionsArgs, agent_config: &AgentConfig) -> Result<()> {
.await;
print_sessions_grouped(&sessions);
}
SessionsCommand::Search { query, limit } => {
SessionsCommand::Search {
query,
limit,
local,
} => {
use std::collections::HashSet;
use xai_grok_shell::session::merge::REMOTE_TIMEOUT;
use xai_grok_shell::session::storage::search::{
Expand All @@ -83,6 +102,16 @@ pub async fn run(args: SessionsArgs, agent_config: &AgentConfig) -> Result<()> {
// Search is the only subcommand that reads the index, so it is the only one to start one
let search = xai_grok_shell::session::storage::search::start_if_enabled(agent_config);

let structured = SessionQuery::parse(&query).is_structured();
if local_only && let Some(by) = search.off_reason() {
anyhow::bail!(
"local session search is off ({by}); enable it to use local or structured search"
);
}
if structured && !local {
eprintln!("Structured query: searching local sessions only.");
}

let req = SessionSearchRequest {
query,
cwd: Some(cwd.to_string_lossy().to_string()),
Expand All @@ -92,10 +121,13 @@ pub async fn run(args: SessionsArgs, agent_config: &AgentConfig) -> Result<()> {
};
let root = grok_home();

let remote_limit = (limit * 3).max(100) as i64;
let remote_limit = i64::try_from(limit.saturating_mul(3).max(100)).unwrap_or(i64::MAX);
let (local_resp, remote_results) = tokio::join!(
execute_search(IndexDecision::settled(&search), &root, &req),
async {
if local_only {
return Vec::new();
}
tokio::time::timeout(
REMOTE_TIMEOUT,
client.search(Some(&req.query), remote_limit),
Expand Down
48 changes: 48 additions & 0 deletions crates/codegen/xai-grok-pager/src/views/session_picker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,14 @@ pub(crate) fn filter_session_entries(
let Some(entries) = entries else {
return vec![];
};
// Literal title/ID matching cannot enforce phrase and content constraints.
// Native structured queries are served exclusively by the deep-search hits.
// Foreign histories keep their existing literal search behavior.
if !source_filter.is_content_search_disabled()
&& xai_grok_shell::session::storage::search::SessionQuery::parse(query).is_structured()
{
return vec![];
Comment on lines +419 to +422

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve foreign literal matches on the All page

When the picker is on SourceFilter::All, this early return suppresses foreign entries as well as native ones because is_content_search_disabled() is true only for External. For example, a Codex session titled title:rust notes previously matched title:rust on the All page but now disappears. Foreign histories are not FTS-indexed, so deep-search results cannot restore it. Apply structured-query suppression per entry to native sources while retaining literal matching for foreign entries.

Useful? React with 👍 / 👎.

}
let q = query.to_lowercase();
entries
.iter()
Expand Down Expand Up @@ -1036,6 +1044,46 @@ mod tests {
matched_fields: vec![],
}
}

#[test]
fn structured_search_only_displays_verified_content_hits() {
// A literal title containing the query is not evidence that the
// content: clause actually matches this session's prompts.
let mut unverified = make_entry("unverified", "repo");
unverified.summary = "content:rust -title:archived".into();
let entries = vec![unverified];
let hits = vec![make_content_hit("verified")];
let items = build_virtual_list(
Some(&entries),
Some(&hits),
"content:rust -title:archived",
SourceFilter::Grok,
);
assert!(matches!(
items.as_slice(),
[PickerItem::Content { hit_index: 0 }]
));
assert!(
build_virtual_list(
Some(&entries),
None,
"content:rust -title:archived",
SourceFilter::Grok,
)
.is_empty()
);
}

#[test]
fn external_history_keeps_literal_search_for_structured_looking_titles() {
let mut external = make_entry("external", "repo");
external.source = "codex".into();
external.summary = "title:rust notes".into();
assert_eq!(
filter_session_entries(Some(&[external]), "title:rust", SourceFilter::External,),
[0]
);
}
/// Grouped entry map places repo headers and resolves mouse-click indices to the correct original session index.
#[test]
fn grouped_entry_map_resolves_correct_session() {
Expand Down
73 changes: 10 additions & 63 deletions crates/codegen/xai-grok-session-search/src/fts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ use std::time::Duration;
use rusqlite::{Connection, OptionalExtension, params};
use xai_sqlite_journal::JournalMode;

use crate::query::SessionQuery;
use crate::recovery;

/// Bump when making breaking schema changes that require dropping and recreating tables, or to force a rebuild of stale index content.
Expand Down Expand Up @@ -439,7 +440,8 @@ impl SessionSearchIndex {
Ok(ids)
}

/// When that intersection matches nothing the query reruns as an OR, so partial matches still show up. A query shaped
/// Plain keywords use AND first, then OR when there are no hits. Explicit
/// phrases, field scopes, and exclusions use a strict plan with no fallback. A query shaped
/// like a session id (a full UUID or a hyphenated hex prefix) matches `session_docs.session_id` directly. FTS only
/// indexes title and content, and a hyphenated UUID `MATCH` looks for tokens that were never indexed.
pub fn query(
Expand All @@ -457,18 +459,21 @@ impl SessionSearchIndex {
}
}

let Some((and_query, or_query)) = Self::build_match_queries(query) else {
let plan = SessionQuery::parse(query);
let Some(primary) = plan.primary_match() else {
Comment on lines +462 to +463

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Check structured syntax before taking the session-ID shortcut

A negative-only hexadecimal query can return precisely the session it appears to exclude. For example, -deadbeef passes is_session_id_like() because that helper strips hyphens, and query_session_id() returns a session such as 12345678-1234-1234-1234-deadbeef0000 before this parser runs. This violates the new negative-only query contract through both CLI and ACP search. Parse before the ID shortcut and restrict that shortcut to unstructured queries.

Useful? React with 👍 / 👎.

return Ok(QueryResult {
results: Vec::new(),
next_offset: None,
total_estimate: Some(0),
});
};

let result = self.run_match_query(&and_query, cwd, limit, offset, include_content)?;
let result = self.run_match_query(primary, cwd, limit, offset, include_content)?;
// Gate the fallback on the total (not the page) so every offset of one logical query is served by the same match string
if result.total_estimate == Some(0) && and_query != or_query {
return self.run_match_query(&or_query, cwd, limit, offset, include_content);
if result.total_estimate == Some(0)
&& let Some(fallback) = plan.fallback_match()
{
return self.run_match_query(fallback, cwd, limit, offset, include_content);
}
Ok(result)
}
Expand Down Expand Up @@ -623,64 +628,6 @@ impl SessionSearchIndex {
total_estimate: Some(total_usize),
})
}

/// Build the AND-joined and OR-joined FTS5 MATCH strings for a query.
///
/// The strings are identical for single-token queries, which lets the caller skip the fallback rerun.
fn build_match_queries(query: &str) -> Option<(String, String)> {
let prefixes: Vec<String> = query
.split_whitespace()
.flat_map(Self::sanitize_token)
.map(Self::token_prefix)
.collect();

if prefixes.is_empty() {
let fallback = query.trim();
if fallback.is_empty() {
return None;
}
let cleaned = fallback.replace('"', "");
let phrase = format!("\"{cleaned}\" *");
return Some((phrase.clone(), phrase));
}

Some((prefixes.join(" AND "), prefixes.join(" OR ")))
}

/// Split a query word on every stripped character instead of gluing: `session_picker.rs` searches as `session_picker` and `rs`.
/// Gluing produced the never-indexed `session_pickerrs`.
/// Fragments with no alphanumeric (`-`, `->`, `_`) are dropped: they tokenize to empty phrases; an empty phrase inside an AND matches nothing.
fn sanitize_token(token: &str) -> impl Iterator<Item = &str> {
token
.split(|c: char| !(c.is_ascii_alphanumeric() || c == '_' || c == '-'))
.filter(|part| part.chars().any(|c| c.is_ascii_alphanumeric()))
}

/// One quoted FTS5 prefix per token, stemmed on the query side only. Plural queries reach singular docs by searching the
/// shorter stem (`sessions` becomes `session*`, `caches` becomes `cach*`). Words shorter than four letters, identifiers
/// with digits/`_`/`-`, and words ending in `ss` (`pass`, `class`) stay exact.
fn token_prefix(token: &str) -> String {
let stem = if token.len() < 4 || !token.chars().all(|c| c.is_ascii_alphabetic()) {
token
} else {
let lower = token.to_ascii_lowercase();
if lower.ends_with("es") {
// Stemming `caches` to `cach` still finds `cache`: the trailing `*` covers the dropped `e`
match token.len().checked_sub(2).and_then(|n| token.get(..n)) {
Some(s) => s,
None => token,
}
} else if lower.ends_with('s') && !lower.ends_with("ss") {
match token.len().checked_sub(1).and_then(|n| token.get(..n)) {
Some(s) => s,
None => token,
}
} else {
token
}
};
format!("\"{stem}\" *")
}
}

/// True when `query` is a UUID or a hyphenated hex prefix long enough to be an intentional session-id search (not a normal keyword).
Expand Down
5 changes: 4 additions & 1 deletion crates/codegen/xai-grok-session-search/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
//!
//! Layering, bottom up:
//!
//! - [`fts`] owns the schema, the BM25 query, and the fenced `meta` writes.
//! - [`query`] parses user syntax into strict or relaxed FTS5 search plans.
//! - [`fts`] owns the schema, executes those plans with BM25 ranking, and fences `meta` writes.
//! - `recovery` classifies an unusable database file, quarantines it, and recreates an empty one; `db` adds the open-and-retry helpers on top.
//! - `doc` turns a session plus its extracted text into an index document.
//! - `bootstrap` runs the full reindex, held to one process at a time by a lease.
Expand All @@ -18,11 +19,13 @@ mod db;
mod doc;
pub mod fts;
mod manager;
pub mod query;
mod recovery;
mod source;

pub use manager::{
SearchIndexManager, SearchIndexStatus, SessionSearchRequest, SessionSearchResponse,
evict_session, execute_search,
};
pub use query::SessionQuery;
pub use source::{ContentExtractor, IndexableSession, SessionSource, SessionSourceFactory};
Loading