Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@
"eslint-plugin-storybook": "^0.11.2",
"gh-pages": "^6.3.0",
"globals": "^15.13.0",
"graphql": "^16.14.2",
"happy-dom": "^20.8.3",
"husky": "^9.1.7",
"jsdom": "^28.1.0",
Expand Down
121 changes: 117 additions & 4 deletions src/api/tiled/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Tiled API client (`tiled`)

A complete client for [Tiled](https://github.com/bluesky/tiled): **all 48 API operations** from
`openapi.json`, the seven auth routes the spec omits, zarr URL builders, and a TanStack Query hook
for every one of them.
`openapi.json`, the seven auth routes the spec omits, zarr URL builders, the experimental
[graph of links](#graph-of-links) over GraphQL, and a TanStack Query hook for every one of them.

Built on the client in
[tiled-viewer-react](https://github.com/bluesky/tiled-viewer-react/tree/main/src/components/Tiled/api)
Expand Down Expand Up @@ -244,6 +244,7 @@ endpoint types.
| Webhooks | list, history | register, delete |
| Server info | about, health, UI settings, metrics | — |
| Auth | whoami | login, logout, API key create/revoke, session refresh/revoke |
| Graph of links | entity, entities, link, links, namespaces, raw `useTiledGraphQLQuery` | entity / link create, update, delete; namespace upsert, delete; raw `useTiledGraphQLMutation` |

### `useTiledDistinctQuery` is the one to know about

Expand Down Expand Up @@ -322,7 +323,7 @@ header.

## Keys and invalidation

Keys are `['tiled', <resource>, <args | null>, { baseUrl, initialPath }]`, with fourteen resources.
Keys are `['tiled', <resource>, <args | null>, { baseUrl, initialPath }]`, with fifteen resources.
The scope is last so prefixes like `['tiled','search']` still match.

**The scope carries `initialPath`, not just `baseUrl`.** Tiled prepends the client's initial path to
Expand Down Expand Up @@ -444,9 +445,121 @@ serve the server's own web UI, and the eight per-file zarr routes (`.zattrs`, `.
`getZarrV3Url` return. Shipping typed fetchers for those would invite a consumer that should have
used a zarr library.

## Graph of links

Tiled's **experimental** entity/link graph (upstream #1387): *entities* — a project, a sample, a
dataset — optionally bound to a catalog node, and *links* `subject —predicate→ object` between them,
with CURIE namespaces for predicates. It is **GraphQL, not REST**, served at `{origin}/api/graphql`
(outside `/api/v1`, like `/healthz`), and only on a catalog-backed server.

**Requires Tiled 0.2.19 or later** (0.2.19rc1+ on PyPI). The graph has been in Tiled since 0.2.15,
but 0.2.15–0.2.18 serve an older shape (`nodeId`, `catalogNodeId`, `accessBlob`) that this client
does not support; against one, entity and link calls reject with an "upgrade the server" error. Built
against bluesky/tiled@849c984a, whose graph code matches v0.2.19rc2 — see
[`graph/README.md`](./graph/README.md).

```ts
import {
bindTiledGraphEntity,
createTiledGraphEntity,
createTiledGraphLink,
upsertTiledGraphNamespace,
} from '@/api/tiled';

await upsertTiledGraphNamespace('schema', 'https://schema.org/');
await upsertTiledGraphNamespace('prov', 'http://www.w3.org/ns/prov#');

const project = await createTiledGraphEntity({ entityType: 'project', name: 'battery-cathodes-2026' });
const sample = await createTiledGraphEntity({ entityType: 'sample', name: 'S001' });
const run = await bindTiledGraphEntity('raw/3f2c9a1e', { entityType: 'dataset' }); // a Bluesky run

await createTiledGraphLink({ subjectId: sample.id, predicate: 'schema:isPartOf', objectId: project.id });
await createTiledGraphLink({ subjectId: run.id, predicate: 'prov:used', objectId: sample.id });
```

### Entities bound to catalog nodes

The server binds an entity to a node through `nodePathParts` but never says which node: entities
expose only `isNodeBound`. Upstream's convention, which its Python client's `bind_entity` follows,
is to also store the node's metadata URL in `uri`. Use `bindTiledGraphEntity` /
`useTiledBindGraphEntityMutation` rather than passing `nodePathParts` yourself. They set both
fields, and they resolve the path against the client's `initialPath`. That matters because
`nodePathParts` counts from the server root while every Finch path is relative to the prefix, so
with `initialPath: 'bl531'` the path `raw/x` is the node `bl531/raw/x`.

```ts
// entity → catalog node
const located = tiledGraphEntityNodePath(run); // { path: 'bl531/raw/3f2c9a1e', sameServer: true }
if (located) await getTiledMetadata(located.path, { pathMode: 'absolute' });

// catalog node → entities
const bound = useTiledGraphNodeEntitiesQuery('raw/3f2c9a1e');
```

`tiledGraphEntityNodePath` returns `null` for an entity with no Tiled metadata `uri`. Its path is
from the server root, hence `pathMode: 'absolute'`. `sameServer: false` means the URL names another
host, usually because the server moved; the path is still the one to try. To backfill an entity
bound without a `uri`, find it with `getTiledGraphNodeEntities(path)` and call
`updateTiledGraphEntity(id, { uri: getTiledGraphNodeUri(path) })`.

The typed hooks select one level — an entity's fields, a link's fields with ids for its ends. For
anything nested, `useTiledGraphQLQuery` sends your own document through the same transport, so a
project's samples come back in one request:

```tsx
const PROJECT_SAMPLES = `query ($id: ID!) {
entity(id: $id) {
name
incomingLinks(predicate: "schema:isPartOf") { subject { id name isNodeBound } }
}
}`;
const project = useTiledGraphQLQuery<{
entity: { name: string; incomingLinks: { subject: { id: string; name: string } }[] } | null;
}>(PROJECT_SAMPLES, { id: projectId });
```

Define the document at module scope — it is part of the cache key. The server caps nesting at depth
10: four `incomingLinks { subject { … } }` hops fit, five are rejected (checked against a live server).

Five things the server does that a client has to know:

- **Failures arrive as HTTP 200.** A resolver error comes back as `{ data, errors }`. Every graph call
rejects whenever `errors` is non-empty, with `error.graphqlErrors` and `error.graphqlCode` (e.g.
`NODE_NOT_FOUND` for a `nodePathParts` that names no node). A server without the graph answers 404,
and the error says so.
- **Reads never answer 401.** The graph filters what a caller may see rather than refusing. With an
access policy configured, an unauthenticated read comes back as empty lists; on a server with
**no** access policy (single-user `--api-key` mode, for one) it returns everything — even though
the same anonymous caller gets a 401 from REST `search`. So an empty graph usually means "check the
key", and a non-empty one proves nothing about authentication. Writes need `write:metadata`.
- **Updates are three-state.** In `updateGraphEntity` / `updateGraphLink`, an omitted field is
untouched and `null` clears it — `{ nodePathParts: null }` detaches an entity from its node. The
client sends exactly what you pass. `properties` cannot be updated at all.
- **Lists are pages without a count.** Every list defaults to `limit: 100`; a page shorter than
`limit` is the only end-of-list signal. Page with `offset`.
- **Nothing is unique yet.** On `main`, two entities can share a type and name, and a failing field in
an aliased multi-mutation nulls the whole `data` after earlier fields have committed. Upstream #1499
adds a uniqueness constraint.

**Keys and invalidation.** Every graph read — typed or raw — keys under one `graph` root, and every
graph write refreshes all of it, because writes ripple: deleting an entity deletes its links, and a
namespace change re-renders every predicate. Deleting a catalog node refreshes it too (the
`structure` bundle includes `graph`), because bound entities are deleted with their node.

**Coverage** is checked against [`graph/schema.graphql`](./graph/schema.graphql), not `openapi.json`:
`TiledGraph.test.tsx` validates every document in [`graph/operations.ts`](./graph/operations.ts)
against the snapshot and fails when the schema has a root field the client does not. Upstream is still
renaming things (#1499: `entityType` → `kind`); `operations.ts` is the only file that spells a field
name, so such a rename is a small edit the test points at. The operations are described in
`TILED_GRAPH_ENDPOINTS`, separate from the spec-checked `TILED_ENDPOINTS`.

Deliberately not here yet: traversal helpers, auto-pagination and batch writes. Each mostly works
around something upstream is likely to change (no counts, no bulk mutation, no uniqueness), so they
wait for the API to settle and for a Finch screen that needs them.

## Not here yet

**Websockets and GraphQL.** Three seams are reserved so they land additively:
**Websockets.** Three seams are reserved so they land additively:

- `apiKeyLocation: 'query'` is implemented already, because a browser websocket handshake cannot set
headers and query-string auth is the only mode that will work. The queue server learned this the
Expand Down
56 changes: 51 additions & 5 deletions src/api/tiled/SKILLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ client/
endpoints/
readEndpoints.ts registry descriptors: info, search, metadata GET, data reads, assets
writeEndpoints.ts registry descriptors: writes, auth, zarr
graphEndpoints.ts graph-of-links descriptors (TILED_GRAPH_ENDPOINTS) — NOT spec routes

graph/
schema.graphql SDL snapshot of the GraphQL graph API — DO NOT hand-edit
operations.ts every GraphQL document; the ONLY file that spells a graph field name
README.md provenance, regeneration, known upstream churn

types/
paths.ts TILED_PATHS (spec paths) + toClientPath + buildPath + completeness assert
Expand All @@ -38,20 +44,21 @@ types/
requestOptions.ts TiledRequestOptions — the transport contract
dataOptions.ts array/table/node option + return maps, and the endpoint/transport split
searchFilters.ts all 15 filters; six values widened to take real values
errors.ts TiledApiError + toTiledApiError
registry.ts descriptor types + payloadAs/numberParam/tupleParam
errors.ts TiledApiError (+ graphqlErrors/graphqlCode) + toTiledApiError
graph.ts graph entity/link/namespace types, inputs, GraphQL envelope
registry.ts descriptor types (REST + graph) + payloadAs/numberParam/tupleParam
common.ts the re-export hub + interceptor types
index.ts barrel

runtime/
clientLike.ts TiledClientLike = Pick<TiledApiClient, ~68> + TILED_CLIENT_LIKE_METHODS
clientLike.ts TiledClientLike = Pick<TiledApiClient, ~82> + TILED_CLIENT_LIKE_METHODS
TiledApiProvider.tsx client context + useTiledApiClient / useTiledApiClientOptional
index.ts barrel

hooks/
types.ts TiledHookError + re-export of the shared Finch option types
errors.ts TiledEndpointUnavailableError
queryKeys.ts 14 roots + key builders; scope LAST, and carries initialPath
queryKeys.ts 15 roots + key builders; scope LAST, and carries initialPath
invalidation.ts bundles + mutation->bundle map + useTiledInvalidate
useTiledClient.ts resolver: provider -> Finch-configured singleton; useTiledQueryScope()
internal/
Expand All @@ -67,6 +74,7 @@ hooks/
managementHooks.ts revisions, assets, webhooks, register, data source, stream — 5q + 6m
infoHooks.ts 5 queries
authHooks.ts 1 query + 6 mutations
graphHooks.ts graph of links — 7 queries + 10 mutations (incl. raw GraphQL, node binding)
typeTests.ts compile-time assertions (tsconfig excludes src/testing, so they live here)
index.ts barrel
```
Expand Down Expand Up @@ -96,6 +104,19 @@ The spec is the source of truth and the coverage test enforces it. Five steps, i

Then the hook: see below.

### Adding a graph operation

The graph is GraphQL, so `openapi.json` and `TILED_PATHS` are not involved. `graph/schema.graphql` is
the source of truth and `TiledGraph.test.tsx` enforces it:

1. **`graph/operations.ts`** — add the named document and its `TILED_GRAPH_OPERATIONS` entry.
2. **`types/graph.ts`** — the result and input types.
3. **`client/TiledApiClient.ts`** — a method that calls `this.graphql(...)` and unwraps its field.
4. **`client/facade.ts`**, **`runtime/clientLike.ts`** (both lists), **`endpoints/graphEndpoints.ts`**.
5. The hook in `hooks/graphHooks.ts` on the `graph` root; mutations invalidate `['graph']`.
6. A playground entry in `components/TiledQueryPlayground/catalog/graph*.tsx` and a row in
`stories/TiledHooks.md`.

## Adding a hook

Hooks are written out explicitly, never generated by a factory — a factory would erase hover types,
Expand All @@ -109,7 +130,7 @@ extra endpoint slot because the client's option types merge endpoint parameters
the hooks do not — they `Omit` the transport keys and recombine in `fetch`. Mutation variables go
through `mutate(variables)`, never the hook.

1. Add the resource to `tiledQueryRoots` + `tiledQueryKeys` if it is not one of the existing fourteen.
1. Add the resource to `tiledQueryRoots` + `tiledQueryKeys` if it is not one of the existing fifteen.
2. Write the hook in the matching `hooks/<domain>Hooks.ts`, with `@param` docs on every parameter.
3. If the hook takes an options object that mixes endpoint and transport fields, key it through a
`keyParts` projection — never the object itself.
Expand Down Expand Up @@ -147,12 +168,37 @@ without one fails CI. The two zarr URL builders are the only entries mapped to `
- `getServerInfo` resolving `null` is `data === null`, not `isError`.
- the three corrected filter parameter names, and the JSON-array encodings.

`TiledGraph.test.tsx`:

- `schema.graphql`'s `Query` / `Mutation` fields equal `TILED_GRAPH_OPERATIONS`, and every document
validates against the schema.
- graph descriptors: one per operation, real client methods, writes destructive except the two creates.
- transport: `POST {origin}/api/graphql`; a 200 with `errors` rejects with `graphqlCode`; 400 and 404
are described; an explicit `null` survives an update while an omitted field is absent; 401 refreshes.
- hooks: id/document guards, one `graph` root, graph writes and `deleteNode` refresh it.

`TiledClient.test.ts`: path encoding, origin derivation, downsampling maths (including the
non-array-structure guard), format resolution, JSON-seq parsing, search option mapping, error
normalisation.

## Gotchas

- **The graph client needs Tiled ≥ 0.2.19.** 0.2.15–0.2.18 serve an older graph API (`nodeId`,
`accessBlob`) on which every entity and link document fails validation. `TILED_GRAPH_MIN_VERSION`
and the fields that first appeared in it live in `graph/operations.ts`; `graphql()` turns those
validation failures into an "upgrade the server" error. Bump both when the target version moves.
- **GraphQL fails with HTTP 200.** A resolver error arrives as `{ data, errors }` with status 200;
`graphql()` rejects whenever `errors` is non-empty. Never branch on status for a graph call.
- **The graph never answers 401.** Graph access control filters results instead of refusing: with an
access policy, an unauthenticated read returns empty lists; on a server *without* one (e.g.
single-user `--api-key`), it returns everything — verified against tiled@849c984a, where REST
`search` answered 401 for the same anonymous caller. Never treat a successful graph read as proof of
authentication, and treat an empty graph as "check the key" before "nothing there".
- **Bind entities to nodes with `bindGraphEntity`, not raw `nodePathParts`.** The server never says
which node an entity is bound to, so the path survives only in `uri`, which `bindGraphEntity` sets.
`nodePathParts` is from the server root; Finch paths are relative to `initialPath`.
- **Graph updates are three-state.** Omitted = untouched, `null` = clear. Pass inputs through untouched;
any helper that maps `undefined` to `null` (or drops `null`) breaks detach.
- **Lists travel two different ways.** Repeated keys for `fields`/`sort`/`column`/`field`/`form_key`/
`metadata`; one JSON array for `keys_filter.keys`, `in.value`, `notin.value`, `specs.*`. Getting it
wrong is a silent no-op or a 500. Both forms are in `client/searchParams.ts` with the evidence.
Expand Down
Loading
Loading