Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,38 @@
universe accepted (1 red), empty stages reported (1 red — the narrow-board
test, added after this disable first stayed green: every earlier fixture
occupied all 21 stages).
- Open (the saved-view half): view definitions as data, and sorting.
Quack has no `ORDER BY` (order is storage order), so a table view sorted
by a property needs its own decision.
- *Done 2026-10-21, see the next section:* the saved-view half.

## Saved views (D-QRY-3, saved-view half)

- `hubspo-query::view::SavedView`: object, filter groups, at most one sort
key, columns. Data only; sharing and column visibility are decided at the
output edge.
- `view::page(view, binder, offset, limit, runner)` returns one page of
ordinals in the view's order without `ORDER BY`:
- no key: `Agg::Rows`, storage order;
- `I32` key: the values at the page's first and last rank by
`Count`-threshold bisection (≤ 33 programs each), then only the band
between them (page + ties at its ends) is materialised and sorted;
- code key: one `GroupReduce Count`, then only codes overlapping the page
are materialised. Codes sort by numeric identity.
- Order: value, ties by ordinal ascending, records without a value last in
both directions.
- `Runner` is the caller's executor (count, rows, groups, code domain,
`i32_at`).
- `tests/view_oracle.rs`: every page (sizes 1, 7, 50, 333) of no-sort, two
`I32` keys (one narrow-range nullable, one full-range with `i32::MIN/MAX`)
and two code keys (one nullable, one with an empty code), both directions,
filtered and not, equals a plain Rust sort over 3,000 records. A page at
offsets 0 / 1,000 / 2,000 costs the same bounded number of programs.
- `hubspo-compat::views::open_view`: a filter or the sort on a hidden
property refuses the view; hidden or unknown columns are dropped.
- Eight disable runs measured red: descending tie-break reversed,
`before` count ignored, nulls dropped, bisection off by one, code order
not reversed, band missing the view's filter, sort left out of the gate,
hidden column kept.
- Open: a page-cost probe at 1M+ rows (the `ORDER BY` question waits on
it), multi-key sorts, view sharing (owner/team/everyone) and persistence.

## Lists (D-QRY-2)

Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
|---|---|---|---|---|
| D-QRY-1 | Filter compiler: HubSpot filter groups (AND within group, OR across groups; operators EQ/NEQ/LT/…/HAS_PROPERTY/CONTAINS_TOKEN/IN/BETWEEN) → Quack program | L | Shipped | 2026-10-14: lance-graph #1546 `lance-graph-relalg` (D-RA-1) normalizes a filter before it runs and answers a contradictory one as `Empty` without a program; D-CMR-1 moves that set algebra into the contract. Adopt it there, not from the prototype crate; `hubspo-store::catalog` + `hubspo-query::filter`; 600 random filter sets equal a plain-loop oracle on mask-risc; #19 |
| D-QRY-2 | Lists: active (incremental on version delta) + static (stored mask); membership history | L | Shipped | `hubspo-query::list` (refresh, static add/remove/members, added/removed); tests vs full recomputation. P-LIST-1 2.6 ms at 10M; P-SPARSE-1 1.93 ms, 257 KiB flat at 10M. Storage of list masks lands with D-STO-1 |
| D-QRY-3 | Saved views + board view (report pivot over pipeline × stage) | M | In progress | board half shipped (#21): `hubspo-query::board` (count + amount per stage as two `GroupReduce` programs); saved-view half open: `SavedView` value type + sort at the edge over existing Quack programs (`Agg::Rows` storage order, `Count` threshold bisection for I32, `GroupReduce Count` for coded lanes); ORDER BY is asked upstream only if a page-cost probe fails |
| D-QRY-3 | Saved views + board view (report pivot over pipeline × stage) | M | In progress | board half shipped (#21): `hubspo-query::board` (count + amount per stage as two `GroupReduce` programs); saved-view half shipped 2026-10-21: `hubspo-query::view` (`SavedView`, `page` with edge sort: storage order, `Count`-threshold bisection for I32, `GroupReduce Count` for codes; ties by ordinal, nulls last), oracle over every page, `hubspo-compat::views::open_view` through the D-IAM-3 gate; 8 disable runs red. Open: page-cost probe at 1M+ rows (ORDER BY asked upstream only if it fails), multi-key sort, sharing, persistence |
| D-QRY-4 | Quack lowering of callcenter's `ParsedQuery` (the upstream DataFusion `parsed_query_to_plan` is an unimplemented stub) | M | Queued | |
| D-QRY-5 | Association-traversing filters ("contacts whose company has an open deal"): Quack `Semijoin` over edge lanes for one hop; multi-hop needs cypher-quack shapes that do not exist yet → upstream request first | L | Queued | 2026-10-20: lance-graph #1632/#1633 (D-SRAM-2 lab spike, `crepe-plan-probe`) measured a semijoin reduction before preparation and an addressed semijoin over attested lanes with masked Lance 13 reads; lab only, not authorization; 2026-10-14: the hop is D-CMR-5, preimage `f⁻¹(S)` under a proven mapping (lance-graph `d-cmr-concept-chain-normalization-v1`), not chained key reads (withdrawn in #1524 / ndarray #359). One hop is ready on the existing Quack `Semijoin`; multi-hop waits on D-CMR-5 rather than on new cypher-quack shapes |
| D-QRY-6 | Reports: single-object, cross-object, funnel, time-series, attribution skeleton on `lance-graph-report` | XL | Queued | 2026-10-21 ruling: also owns forecast types (`crm-forecast-types`, 2 functions) and reads D-FLOW-2's custom behavioural events. |
Expand Down
3 changes: 3 additions & 0 deletions crates/hubspo-compat/src/contacts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,9 @@ pub enum SearchRefusal {
SchemaMismatch,
/// The request holds no read on the concept.
NoRead,
/// The view's object is not the concept the access was compiled for, or
/// its table resolves to no concept.
ObjectMismatch,
}

/// Check a search's filter groups and sort properties against the request's
Expand Down
3 changes: 3 additions & 0 deletions crates/hubspo-compat/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,8 @@
//! - [`contacts`]: the first authorized output path (D-IAM-3): one contact
//! by id, and the gate a search's filters and sorts pass before they run.
//! The REST wire format around it lands with D-API-1.
//! - [`views`]: a saved view (D-QRY-3) through the same gate: a hidden
//! filter or sort refuses it, a hidden column is dropped.

pub mod contacts;
pub mod views;
223 changes: 223 additions & 0 deletions crates/hubspo-compat/src/views.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
//! A saved view through the D-IAM-3 gate.
//!
//! A view is shared data: whoever opens it gets its filters, sort and
//! columns. The request's access decides what that may reveal:
//!
//! - a filter or the sort on a property the request may not read refuses
//! the whole view, since which records match (or their order) would leak
//! the hidden values;
//! - a column the request may not read is dropped from the output, the same
//! way [`crate::contacts::lookup`] drops a hidden property: absent, not
//! blanked.

use crate::contacts::{SearchRefusal, check_search};
use hubspo_iam::access::RequestAccess;
use hubspo_iam::fields::{Schema, Stamp};
use hubspo_query::view::SavedView;

/// Check `view` against the request's access; on success, the columns the
/// request may see, in the view's order.
///
/// `table_concept` is the store's answer to which concept a table holds.
/// The view's table must resolve to exactly the concept `access` was
/// compiled for, or the view is refused before anything else is read: an
/// unfiltered view of another object would otherwise pass every property
/// check and be paged against a concept nobody authorized.
///
/// # Errors
///
/// See [`SearchRefusal`].
pub fn open_view(
access: &RequestAccess,
current: &Stamp,
schema: &Schema,
view: &SavedView,
table_concept: impl Fn(&str) -> Option<u16>,
) -> Result<Vec<&'static str>, SearchRefusal> {
if table_concept(&view.object) != Some(access.stamp().concept) {
return Err(SearchRefusal::ObjectMismatch);
}
let sorts: Vec<&str> = view.sort.iter().map(|s| s.property.as_str()).collect();
check_search(access, current, schema, &view.groups, &sorts)?;
Comment thread
AdaWorldAPI marked this conversation as resolved.
let projection = access.projection(current).map_err(SearchRefusal::Refused)?;
Ok(view
.columns
.iter()
.filter_map(|name| {
let ordinal = schema.ordinal(name)?;
let p = schema.property(ordinal)?;
projection.admits(current, schema, name).then_some(p.name)
})
.collect())
}

#[cfg(test)]
mod tests {
use super::*;
use hubspo_generated::standard_properties::{CONTACT, PACKED_RUN_POSITION, Slot};
use hubspo_iam::fields::{
OrdinalSet, PolicyVersion, Refusal, RoleVersion, SchemaVersion, SeatVersion,
};
use hubspo_iam::scope::{OwnerScope, RowScope};
use hubspo_iam::tenant::{
Assignee, PermissionSet, PermissionSetId, Seat, TenantId, TenantPolicy,
};
use hubspo_port::ROLE_VIEW_ONLY;
use hubspo_query::filter::{FilterGroup, Operator, PropertyFilter};
use hubspo_query::view::{Direction, Sort};
use lance_graph_contract::class_view::WideFieldMask;
use lance_graph_contract::rbac::{ClassGrant, OpMask};
use ogar_dir_core::Guid128;
use ogar_vocab::class_ids;

/// The store's table → concept answer in these tests.
fn tables(name: &str) -> Option<u16> {
match name {
"contacts" => Some(class_ids::CRM_CONTACT),
"deals" => Some(class_ids::CRM_DEAL),
_ => None,
}
}

const T: TenantId = TenantId(3);
const C: u16 = class_ids::CRM_CONTACT;

fn contact() -> Schema {
Schema {
concept: C,
version: SchemaVersion(1),
properties: CONTACT,
}
}

fn seat() -> Seat {
Seat {
version: SeatVersion(1),
concepts: vec![C],
actions: Vec::new(),
}
}

/// Reads email, firstname and city.
fn access() -> (RequestAccess, Stamp) {
let lanes = CONTACT.iter().filter_map(|p| match p.slot {
Slot::Lane(n) => Some(n),
Slot::Packed => None,
});
let every = WideFieldMask::from_positions(
&std::iter::once(PACKED_RUN_POSITION)
.chain(lanes)
.collect::<Vec<_>>(),
);
let mut props = OrdinalSet::new();
for n in ["email", "firstname", "city"] {
props.insert(contact().ordinal(n).unwrap());
}
let mut p = TenantPolicy::new(T, PolicyVersion(1));
p.add_set(
PermissionSetId(1),
PermissionSet {
grants: vec![ClassGrant::new(C, OpMask::READ)],
scopes: vec![(C, OwnerScope::Everything)],
fields: vec![(C, every)],
properties: vec![(C, props)],
..PermissionSet::default()
},
)
.unwrap();
p.assign(Assignee::Role(ROLE_VIEW_ONLY), PermissionSetId(1))
.unwrap();
let a = RequestAccess::compile(
&p,
T,
&seat(),
&Guid128([1; 16]),
&[ROLE_VIEW_ONLY],
RoleVersion(1),
&contact(),
|_| RowScope::All,
)
.unwrap();
let now = RequestAccess::current_stamp(&p, &contact(), &seat(), RoleVersion(1));
(a, now)
}

fn view(filter_on: &str, sort_on: &str, columns: &[&str]) -> SavedView {
SavedView {
object: "contacts".into(),
groups: vec![FilterGroup {
filters: vec![PropertyFilter::new(filter_on, Operator::Eq, "x")],
}],
sort: Some(Sort {
property: sort_on.into(),
direction: Direction::Ascending,
}),
columns: columns.iter().map(|c| (*c).to_string()).collect(),
}
}

#[test]
fn a_hidden_column_is_dropped_and_the_rest_keep_their_order() {
let (a, now) = access();
let v = view(
"email",
"city",
&["city", "date_of_birth", "email", "no_such", "firstname"],
);
assert_eq!(
open_view(&a, &now, &contact(), &v, tables),
Ok(vec!["city", "email", "firstname"])
);
}

#[test]
fn a_view_of_another_or_unknown_object_is_refused() {
let (a, now) = access();
// No filter, no sort: nothing for the property gate to catch.
let mut v = SavedView {
object: "deals".into(),
columns: vec!["email".into()],
..SavedView::default()
};
assert_eq!(
open_view(&a, &now, &contact(), &v, tables),
Err(SearchRefusal::ObjectMismatch)
);
v.object = "no_such_table".into();
assert_eq!(
open_view(&a, &now, &contact(), &v, tables),
Err(SearchRefusal::ObjectMismatch)
);
v.object = "contacts".into();
assert_eq!(
open_view(&a, &now, &contact(), &v, tables),
Ok(vec!["email"])
);
}

#[test]
fn a_filter_or_sort_on_a_hidden_property_refuses_the_view() {
let (a, now) = access();
let dob = contact().ordinal("date_of_birth").unwrap();
assert_eq!(
open_view(
&a,
&now,
&contact(),
&view("date_of_birth", "city", &["email"]),
tables
),
Err(SearchRefusal::Refused(Refusal::Hidden(dob)))
);
assert_eq!(
open_view(
&a,
&now,
&contact(),
&view("email", "date_of_birth", &["email"]),
tables
),
Err(SearchRefusal::Refused(Refusal::Hidden(dob)))
);
}
}
3 changes: 3 additions & 0 deletions crates/hubspo-query/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,12 @@
//! mask programs.
//! - [`board`] (D-QRY-3, board half): count and amount per pipeline stage
//! as two grouped programs.
//! - [`view`] (D-QRY-3, saved-view half): a saved view as data and one page
//! of it in sort order, sorted at the edge over Quack programs.
//!
//! [`Filter`]: lance_graph_quack::Filter

pub mod board;
pub mod filter;
pub mod list;
pub mod view;
Loading
Loading