Skip to content

Commit 658d863

Browse files
committed
fix(ingest): correct --write-mode to the real load-mode vocabulary
The --write-mode flag hardcoded value_parser = ["replace", "upsert"], which contradicted what the CLI itself shows and what the service accepts: - `ingest sources fields <family>` renders the server-reported per-family write_modes (e.g. filesystem: replace, append), so the CLI advertised `append` while the create flag rejected it with 'invalid value append'. - The real service-wide load-mode vocabulary is replace/append (keyless) plus the key-based upsert/update/delete (confirmed against the OpenAPI spec and the dlt worker, which maps upsert -> dlt merge/upsert). The old pair both omitted valid modes (append, update, delete) and offered `upsert` as if universal, though it is key-gated. Correct the enum to the full set so clap still catches typos up front while accepting every mode a family may advertise; per-family/per-table support is still enforced by the service. Fix the SKILL.md note likewise (it wrongly said upsert needs a continuous bucket ingest; it needs a declared key). Closes #278
1 parent 261848e commit 658d863

2 files changed

Lines changed: 19 additions & 11 deletions

File tree

‎skills/hotdata/SKILL.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,9 @@ hotdata ingest create --datasource-id ds_01J --type one-time \
301301
# selector.json is family-specific (what subset to read) — its fields, and the
302302
# write modes this family accepts: hotdata ingest sources fields <family>.
303303
# destination.json is {"database_id", "schema", "table", "write_mode"} —
304-
# write_mode: replace | upsert (upsert needs a continuous bucket ingest).
304+
# write_mode (default replace): replace|append work on any table; the key-based
305+
# modes upsert|update|delete need the destination table to have a declared key.
306+
# `hotdata ingest sources fields <family>` reports which modes the family accepts.
305307
# Selector and destination are both IMMUTABLE after creation.
306308
# CREATE STARTS NOTHING, for every type. It returns no run id, and
307309
# `ingest logs <id>` is EMPTY until the scheduler claims the ingest — normal,
@@ -328,7 +330,9 @@ hotdata ingest create --source "prod postgres" --table orders --schema public \
328330
# --limit N stop after N source rows
329331
# Destination flags instead of --destination:
330332
# --database-id (required) --dest-table (defaults to the single --table)
331-
# --dest-schema (default public) --write-mode (default replace)
333+
# --dest-schema (default public)
334+
# --write-mode replace|append|upsert|update|delete (default replace; upsert/
335+
# update/delete need the destination table to have a declared key)
332336

333337
hotdata ingest create --datasource-id ds_01J --database-id db_123 \
334338
--sql "SELECT id, status FROM public.orders WHERE status = 'open' LIMIT 1000"

‎src/commands/ingest.rs‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -238,15 +238,19 @@ pub enum IngestCommands {
238238
#[arg(long = "dest-schema")]
239239
dest_schema: Option<String>,
240240

241-
/// How each run writes (default: replace). `upsert` needs a family
242-
/// whose load path stamps a row key — today that is a continuous bucket
243-
/// ingest, and `hotdata ingest sources fields <family>` reports which modes
244-
/// a family accepts for which type.
245-
///
246-
/// The two listed are the two the destination accepts anywhere. Offering
247-
/// a third would be offering a request that is refused on arrival, which
248-
/// costs the user a round trip to learn what `--help` could have said.
249-
#[arg(long = "write-mode", value_parser = ["replace", "upsert"])]
241+
/// How each run writes (default: `replace`). `replace` and `append` work
242+
/// on any destination table; the key-based modes `upsert`, `update`, and
243+
/// `delete` match rows by the table's declared key and are rejected on a
244+
/// keyless table. This list is the service-wide vocabulary — clap catches a
245+
/// typo up front — but which of these a *given* family or table accepts is
246+
/// reported by `hotdata ingest sources fields <family>`, the authoritative
247+
/// source; a mode the family or table does not support is refused on
248+
/// arrival. (The old list `[replace, upsert]` both omitted `append` — which
249+
/// families advertise — and offered `upsert` as if universal: see #278.)
250+
#[arg(
251+
long = "write-mode",
252+
value_parser = ["replace", "append", "upsert", "update", "delete"]
253+
)]
250254
write_mode: Option<String>,
251255

252256
/// Schedule as JSON (inline, @file.json, or @-):

0 commit comments

Comments
 (0)