Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion skills/hotdata/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ Returns workspaces with `current`, `public_id`, `name`, `active`, `favorite`, `p
```
hotdata databases list [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases count [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases create [--name <display_name>] [--catalog <alias>] [--table <table> ...] [--schema public] [--expires-at <duration|timestamp>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases create [--name <display_name>] [--catalog <alias>] [--table <table> ...] [--schema public] [--expires-at <duration|timestamp>] [--if-not-exists] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases fork [<id>] [--name <display_name>] [--expires-at <duration|timestamp>] [--description <text>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases lineage [<id>] [--forks-limit <n>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases use <id>
Expand All @@ -130,6 +130,7 @@ hotdata databases tables remove <table> [--database <id>] [--schema public] [--w
- `list` — all instant databases in the workspace. Current database is marked with `*` under the CURRENT column; CREATED shows when each database was made.
- `count` — the total number of instant databases in the workspace, across **all** pages (`list` shows one page). Prints a bare integer by default so it drops straight into scripts (`$(hotdata databases count)`); `--output json|yaml` render `{"count": N}` / `count: N`.
- `create` — creates a new instant database. `--name` is an optional human-readable display name. `--catalog` sets the SQL alias used in queries (`SELECT … FROM <catalog>.schema.table`); must be `[a-z_][a-z0-9_]*`. `--expires-at` accepts relative durations (`24h`, `7d`, `90m`) or an RFC 3339 timestamp; omitting means no expiry. Repeat `--table` to declare tables up front. **Does not switch the current database** — run `hotdata databases use <id>` with the printed id (or pass `-d <id>`) to work in it.
- `create --if-not-exists --name <name>` — get-or-create: when a database already carries that name it is returned instead of a new one being created, so a script can bind to its database on every run without looking it up first. The table output says `Database already exists` in that case, and `-o json`/`yaml` carry `created: true|false`. Requires `--name`; names are otherwise not unique, so without the flag a second `create` with the same name makes a second database.
- `fork` — creates a new instant database that is an independent deep copy of an existing one (same schemas, tables, and data); the source is left unchanged and the two diverge freely afterwards. The source defaults to the current database; pass the database `<id>` to fork another. `--name` defaults to `<source>-fork` (so the two stay distinguishable in `list`); `--expires-at` accepts a relative duration or RFC 3339 timestamp, and when omitted a still-future source expiry is carried over. `--description` records why the fork was taken (for example `"what-if on Q3 pricing"`); it can be set only at fork time. **Does not switch the current database** — run `hotdata databases use <fork_id>` to work in the fork, or pass `-d <fork_id>` per command. The fork answers to the **same catalog alias** as its source inside its own scope; catalogs attached to the source are **re-attached** to the fork, but indexes are **not** carried over. Only databases created with the current (DuckLake) storage engine can be forked — older parquet-backed databases return an error. The fork's output (and its `databases <id>` view) records **`forked_from`** provenance: source id, the source's name at fork time, the copied snapshot, when, and the description if one was given.
- `lineage` — renders a database's **whole fork family tree**, walked from the root down (defaults to the current database; one lineage request per database, fine for the small families forks produce in practice): every reachable generation, with the queried database marked `← this database`. A fork taken with `--description` shows the first line of it on its row; `-o json`/`yaml` carry the full text. Lineage is a historical record, not a live link — the databases stay independent, and a **deleted** generation stays in the chain (marked `deleted`), though its own fork list can't be enumerated: such branches end with `⋯ forks unknown`. Forks made before the server recorded lineage carry none. `--forks-limit <n>` pages each database's direct-fork list (server clamps to 1–100); a truncated branch closes with `⋯ N more`. `-o json`/`yaml` return `{database_id, root_id, tree}` with a recursive `tree` node.
- `use` — saves the database as the current database (accepts an id, catalog alias, or name; prefer the id). Subsequent `query`, `databases tables`, and `databases context` commands use it automatically. This is the **only** command that changes the current database.
Expand Down
103 changes: 89 additions & 14 deletions src/commands/databases.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@ pub enum DatabasesCommands {
#[arg(long = "attach")]
attach: Vec<String>,

/// Get-or-create: when a database with this `--name` already exists,
/// return it instead of creating another. Lets a script bind to its
/// database on every run without looking it up first. Requires `--name`.
#[arg(long, requires = "name")]
if_not_exists: bool,

/// Output format
#[arg(long = "output", short = 'o', default_value = "table", value_parser = ["table", "json", "yaml"])]
output: String,
Expand Down Expand Up @@ -627,6 +633,10 @@ struct CreateDatabaseResponse {
/// responses never carry it.
#[serde(default, skip_serializing_if = "Option::is_none")]
forked_from: Option<ForkedFrom>,
/// Only with `--if-not-exists`: `true` when this call created the database,
/// `false` when an existing one was returned.
#[serde(default, skip_serializing_if = "Option::is_none")]
created: Option<bool>,
}

#[derive(Deserialize)]
Expand Down Expand Up @@ -824,13 +834,20 @@ pub fn create_database_request(
schema: &str,
tables: &[String],
expires_at: Option<&str>,
if_not_exists: bool,
) -> serde_json::Value {
let mut req = serde_json::Map::new();

if let Some(n) = name {
req.insert("name".to_string(), serde_json::Value::String(n.to_string()));
}

// Omitted rather than `false`: the field is new on the server, and an older
// deployment rejects unknown keys it would otherwise never see.
if if_not_exists {
req.insert("if_not_exists".to_string(), serde_json::Value::Bool(true));
}

if let Some(c) = catalog {
req.insert(
"default_catalog".to_string(),
Expand Down Expand Up @@ -933,9 +950,15 @@ fn create_database_typed_request(
schema: &str,
tables: &[String],
expires_at: Option<&str>,
if_not_exists: bool,
) -> hotdata::models::CreateDatabaseRequest {
serde_json::from_value(create_database_request(
name, catalog, schema, tables, expires_at,
name,
catalog,
schema,
tables,
expires_at,
if_not_exists,
))
.expect("create_database_request always emits a valid CreateDatabaseRequest body")
}
Expand Down Expand Up @@ -2072,11 +2095,13 @@ pub fn create(
tables: &[String],
expires_at: Option<&str>,
attach: &[String],
if_not_exists: bool,
format: &str,
) {
use crossterm::style::Stylize;

let request = create_database_typed_request(name, catalog, schema, tables, expires_at);
let request =
create_database_typed_request(name, catalog, schema, tables, expires_at, if_not_exists);

let api = Api::new(Some(workspace_id));
let resp = if format == "table" {
Expand All @@ -2090,6 +2115,11 @@ pub fn create(
}
.unwrap_or_else(|e| e.exit());

// With `--if-not-exists` the server says whether it created or found the
// database; `created` is absent on older servers, which never return an
// existing one here, so absent reads as created.
let created = resp.created.flatten().unwrap_or(true);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Skip the --attach loop below when created is false, or document the rerun behavior (not blocking).

With --if-not-exists --attach <conn>, every rerun calls attach_connection on the existing database again. If the server rejects a duplicate attach, each rerun prints warning: could not attach. That warning defeats the "bind on every run" use case in SKILL.md. Also state in SKILL.md that --catalog, --table, and --expires-at are ignored when an existing database is returned.


// Attach requested connection catalogs onto the fresh database so a single
// `databases create --attach …` lands a cross-source-ready context. A failed
// attach is surfaced but non-fatal: the database exists and is usable.
Expand Down Expand Up @@ -2122,13 +2152,19 @@ pub fn create(
// Never set on a plain create; mapped anyway so the shape can't drift
// from fork's if the server ever returns an existing fork here.
forked_from: resp.forked_from.flatten().map(|f| ForkedFrom::from(*f)),
// Only meaningful with `--if-not-exists`, where a script branches on it.
created: if if_not_exists { Some(created) } else { None },
};

match format {
"json" => println!("{}", serde_json::to_string_pretty(&result).unwrap()),
"yaml" => print!("{}", serde_yaml::to_string(&result).unwrap()),
"table" => {
println!("{}", "Database created".green());
if created {
println!("{}", "Database created".green());
} else {
println!("{}", "Database already exists".green());
}
if let Some(n) = &result.name {
println!("name: {}", n.clone().cyan());
}
Expand Down Expand Up @@ -2236,6 +2272,7 @@ pub fn fork(
default_connection_id: resp.default_connection_id,
expires_at: resp.expires_at.flatten(),
forked_from: resp.forked_from.flatten().map(|f| ForkedFrom::from(*f)),
created: None,
};

match format {
Expand Down Expand Up @@ -3229,9 +3266,23 @@ mod tests {
);
}

#[test]
fn create_database_request_sends_if_not_exists_only_when_set() {
let req = create_database_request(Some("app"), None, "public", &[], None, true);
assert_eq!(req["if_not_exists"], serde_json::json!(true));
let req = create_database_request(Some("app"), None, "public", &[], None, false);
assert!(
req.get("if_not_exists").is_none(),
"an older server rejects unknown keys, so false is omitted, not sent"
);
// The typed request carries it through to the SDK model.
let typed = create_database_typed_request(Some("app"), None, "public", &[], None, true);
assert_eq!(typed.if_not_exists, Some(true));
}

#[test]
fn create_database_request_declares_schema_without_name_or_tables() {
let req = create_database_request(None, None, "public", &[], None);
let req = create_database_request(None, None, "public", &[], None, false);
assert_eq!(
req,
serde_json::json!({"schemas": [{"name": "public"}]}),
Expand All @@ -3241,7 +3292,7 @@ mod tests {

#[test]
fn create_database_request_declares_custom_schema_without_tables() {
let req = create_database_request(None, None, "analytics", &[], None);
let req = create_database_request(None, None, "analytics", &[], None, false);
assert_eq!(req["schemas"][0]["name"], "analytics");
// An empty schema declaration omits `tables` entirely — the server
// treats that as "declare the schema, add tables later".
Expand All @@ -3252,7 +3303,7 @@ mod tests {
fn create_database_request_typed_round_trip_without_tables() {
// The typed builder deserializes the JSON body and panics on a shape the
// SDK model rejects, so the tables-less schema declaration must survive it.
let req = create_database_typed_request(None, None, "public", &[], None);
let req = create_database_typed_request(None, None, "public", &[], None, false);
let schemas = req.schemas.expect("schemas is always sent");
assert_eq!(schemas.len(), 1);
assert_eq!(schemas[0].name, "public");
Expand All @@ -3261,7 +3312,7 @@ mod tests {

#[test]
fn create_database_request_includes_name() {
let req = create_database_request(Some("jaffle_shop"), None, "public", &[], None);
let req = create_database_request(Some("jaffle_shop"), None, "public", &[], None, false);
assert_eq!(req["name"], "jaffle_shop");
assert_eq!(req["schemas"][0]["name"], "public");
}
Expand All @@ -3274,6 +3325,7 @@ mod tests {
"public",
&["orders".to_string(), "customers".to_string()],
None,
false,
);
assert_eq!(req["schemas"][0]["name"], "public");
assert_eq!(req["schemas"][0]["tables"][0]["name"], "orders");
Expand All @@ -3282,7 +3334,14 @@ mod tests {

#[test]
fn create_database_request_schemas_without_name() {
let req = create_database_request(None, None, "analytics", &["events".to_string()], None);
let req = create_database_request(
None,
None,
"analytics",
&["events".to_string()],
None,
false,
);
assert!(req.get("name").is_none());
assert_eq!(req["schemas"][0]["name"], "analytics");
}
Expand All @@ -3292,7 +3351,14 @@ mod tests {
// Every --table dot-qualified elsewhere: --schema still has to be
// declared, or a later bare-name declaration into it fails the same way
// an empty --table list used to.
let req = create_database_request(None, None, "public", &["raw.orders".to_string()], None);
let req = create_database_request(
None,
None,
"public",
&["raw.orders".to_string()],
None,
false,
);
assert_eq!(req["schemas"][0]["name"], "public");
assert!(
req["schemas"][0].get("tables").is_none(),
Expand All @@ -3312,6 +3378,7 @@ mod tests {
"public",
&["orders".to_string(), "public.customers".to_string()],
None,
false,
);
assert_eq!(req["schemas"].as_array().unwrap().len(), 1);
assert_eq!(req["schemas"][0]["name"], "public");
Expand All @@ -3321,13 +3388,13 @@ mod tests {

#[test]
fn create_database_request_includes_expires_at_when_provided() {
let req = create_database_request(None, None, "public", &[], Some("24h"));
let req = create_database_request(None, None, "public", &[], Some("24h"), false);
assert_eq!(req["expires_at"], "24h");
}

#[test]
fn create_database_request_omits_expires_at_when_none() {
let req = create_database_request(None, None, "public", &[], None);
let req = create_database_request(None, None, "public", &[], None, false);
assert!(req.get("expires_at").is_none());
}

Expand All @@ -3343,6 +3410,7 @@ mod tests {
"raw.raw_customers".to_string(),
],
None,
false,
);
// bare "orders" → default schema "public"
assert_eq!(req["schemas"][0]["name"], "public");
Expand Down Expand Up @@ -4331,14 +4399,21 @@ mod tests {
"public",
&["gdp".to_string()],
None,
))
false,
))
Comment on lines +4402 to +4403

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: Indent these lines to match the other arguments (not blocking). rustfmt skips this chain because line 4394 exceeds the width limit, so it will not fix the indentation.

Suggested change
false,
))
false,
))

.unwrap(),
))
.create();

let api = Api::test_new(&server.url(), "k", Some("ws-test"));
let body =
create_database_request(Some("mydb"), None, "public", &["gdp".to_string()], None);
let body = create_database_request(
Some("mydb"),
None,
"public",
&["gdp".to_string()],
None,
false,
);
let (status, resp_body) = api.post_raw("/databases", &body).unwrap();
assert_eq!(status.as_u16(), 201);
let parsed: CreateDatabaseResponse = serde_json::from_str(&resp_body).unwrap();
Expand Down
2 changes: 2 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,7 @@ fn main() {
tables,
expires_at,
attach,
if_not_exists,
output,
}) => databases::create(
&workspace_id,
Expand All @@ -264,6 +265,7 @@ fn main() {
&tables,
expires_at.as_deref(),
&attach,
if_not_exists,
&output,
),
Some(DatabasesCommands::Fork {
Expand Down
Loading