Skip to content

Commit 230b564

Browse files
committed
docs: correct format and declaration claims the code no longer matches
A deliberate sweep for parquet-only language, after finding two stale spots by accident. README's command table and the workflow decision guide still described `databases load` as parquet-only and instant databases as "parquet files you own". Both now name csv, newline-delimited json, and parquet, and the table lists `databases tables add`. The instant-database workflow carried a callout that was materially wrong after this branch: it warned that loading into an undeclared table recreates the database and changes its `id`, and advised against caching ids across loads. The load declares a missing table and schema in place, so the id and the other tables survive. The workflow now shows the declare-a-key step and records that a key cannot be added later. Two code comments justified themselves by the removed delete+recreate path: the raw JSON create-request builder ("the delete+recreate path still consumes the raw JSON form, so the JSON builder stays") and `delete_raw`'s doc. Both are still used for other reasons, so only the reasons are corrected.
1 parent 21b093b commit 230b564

4 files changed

Lines changed: 34 additions & 20 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,8 @@ The full command surface. The top level has nine groups — `auth`, `workspaces`
166166
| `databases use` | Set the current (default) database |
167167
| `databases unset` | Clear the current database |
168168
| `databases remove` | Delete a database and all its tables |
169-
| `databases load` | Load a parquet file or saved result into a table (replace, or `--append`) |
169+
| `databases load` | Load a csv/json/parquet file or saved result into a table (`--mode replace\|append\|delete\|update\|upsert`) |
170+
| `databases tables add` | Declare a table with its key and storage layout |
170171
| `databases tables list` | List tables in a database |
171172
| `databases tables show` | Show column definitions for a table |
172173
| `databases tables load` | Load parquet/result into a table (replace, or `--append`) |

‎skills/hotdata/references/WORKFLOWS.md‎

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The `hotdata` skill is always loaded first (auth and workspace setup). The three
1111
| User goal | Skill | Key commands |
1212
|-----------|--------|----------------|
1313
| Login, workspaces, datasources, tables, context | **`hotdata`** | `auth`, `workspaces`, `ingest sources`, `ingest`, `databases tables`, `databases context` |
14-
| Load parquet files into an instant database | **`hotdata`** | `databases create` + `databases load` |
14+
| Load csv/json/parquet files into an instant database | **`hotdata`** | `databases create` + `databases tables add` + `databases load` |
1515
| SQL analytics, aggregations, history, Chain | **`hotdata-analytics`** (`subskills/analytics/SKILL.md`) | `query`, `databases queries`, `databases results` |
1616
| BM25 / vector search, retrieval indexes | **`hotdata-search`** (`subskills/search/SKILL.md`) | `search`, `search create`, `search embeddings` |
1717
| Geospatial / PostGIS-style SQL | **`hotdata-geospatial`** (`subskills/geospatial/SKILL.md`) | `query` with `ST_*`, WKB columns |
@@ -94,33 +94,47 @@ A `hotdata query` runs inside **one** instant database; its scope sees that data
9494

9595
| | **Instant databases** |
9696
|---|------------------------|
97-
| **Best for** | Parquet files you own; catalog-style `alias.schema.table` |
97+
| **Best for** | Files you own (csv, newline-delimited json, parquet); catalog-style `alias.schema.table` |
9898
| **SQL prefix** | `<catalog>.<schema>.<table>` where catalog = `--catalog` alias |
9999
| **CLI** | `hotdata databases create --catalog` + `databases load` |
100-
| **Declare schema up front** | Yes — `--table` on create (auto-declared on first `databases load`) |
101-
| **Parquet file uploads** | `databases load --file` / `--url` / `--upload-id` |
102-
| **Refresh** | Replace via `databases load` again |
100+
| **Declare schema up front** | Optional — the load declares a missing table/schema. Declare with `databases tables add --key` when you need the keyed load modes; a key cannot be added later |
101+
| **File uploads** | `databases load --file` / `--url` / `--upload-id`; format from the extension, or `--format` |
102+
| **Refresh** | `databases load` again — `--mode replace` (default) or `append`, or `delete`/`update`/`upsert` against a declared key |
103103

104-
**Rule of thumb:** Parquet files you control as **`mydb.public.orders`** → **instant databases**.
104+
**Rule of thumb:** Files you control as **`mydb.public.orders`** → **instant databases**.
105105

106-
### Workflow: instant database (parquet)
106+
### Workflow: instant database (file upload)
107107

108108
1. Create the database with a catalog alias:
109109

110110
```bash
111111
hotdata databases create --catalog sales
112112
```
113113

114-
2. Load parquet per table (tables are auto-declared if needed):
114+
2. Load a file per table. A missing table or schema is declared as part of the
115+
load, and the format comes from the file's extension:
115116

116117
```bash
117-
hotdata databases load --catalog sales --table orders --file ./orders.parquet
118+
hotdata databases load --catalog sales --table orders --file ./orders.csv
118119
hotdata databases load --catalog sales --table customers --url https://example.com/customers.parquet
119120
```
120121

121-
> Auto-declaring a *new* table recreates the database (no add-table API), which **changes its `id`** — the id returned by `databases create` goes stale after the next `load` of an undeclared table. Declare tables up front (`databases create --table orders --table customers`) to avoid the recreate, and don't cache ids across loads: re-read the current id from `databases list` at time of use. (Selection is still always by id — names and catalogs are not unique.)
122+
> The database keeps its `id` and its other tables across a load into an
123+
> undeclared table — nothing is recreated. Selection is still always by id,
124+
> since names and catalogs are not unique.
122125
123-
3. Confirm and query:
126+
3. Declare the table yourself only when you need something the load cannot
127+
infer — a key, a sort order, or partitioning:
128+
129+
```bash
130+
hotdata databases tables add orders --key order_id --sorted-by created_at=desc
131+
hotdata databases load --catalog sales --table orders --file ./changed.csv --mode upsert
132+
```
133+
134+
> A key is fixed when the table is declared and **cannot be added later**, so
135+
> declare it before the first load if you will need `delete`/`update`/`upsert`.
136+
137+
4. Confirm and query:
124138

125139
```bash
126140
hotdata databases tables list

‎src/client/sdk.rs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -852,10 +852,10 @@ impl Api {
852852
/// `Configuration`, returning the raw status + body text.
853853
///
854854
/// The seam's DELETE counterpart to [`post_raw`](Self::post_raw): used by
855-
/// `databases.rs`, where the delete bodies feed the same CLI-side
856-
/// `(status, body)` control flow as the old raw `delete_raw` (e.g. the
857-
/// delete+recreate path inspects the failure body), so non-success is
858-
/// returned as `Ok((status, body))` rather than an error.
855+
/// `databases.rs`, where a delete's response body feeds the same CLI-side
856+
/// `(status, body)` control flow the raw calls used, so non-success is
857+
/// returned as `Ok((status, body))` rather than an error and the caller
858+
/// renders the server's own message.
859859
pub fn delete_raw(&self, path: &str) -> Result<(reqwest::StatusCode, String), ApiError> {
860860
let cfg = self.client.configuration();
861861
let url = format!("{}/v1{path}", cfg.base_path);

‎src/commands/databases.rs‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -877,10 +877,9 @@ pub fn fork_database_request(
877877
}
878878

879879
/// Build the typed `CreateDatabaseRequest` for the SDK's `databases().create`
880-
/// handle, reusing [`create_database_request`] as the single source of truth for
881-
/// the body shape. The delete+recreate path still consumes the raw JSON form
882-
/// (it inspects raw error bodies), so the JSON builder stays; this just adapts
883-
/// it for the typed call sites.
880+
/// handle by adapting [`create_database_request`], which stays the single
881+
/// source of truth for the body shape — the schema-seeding rules it encodes are
882+
/// specified against the JSON form in this module's tests.
884883
fn create_database_typed_request(
885884
name: Option<&str>,
886885
catalog: Option<&str>,

0 commit comments

Comments
 (0)