Skip to content

Commit b341018

Browse files
korenmiklosclaude
andcommitted
Revise M15 plan: tempfile as tables in _stata_temp schema, preserve as checkpoint
tempfile → CREATE TABLE in _stata_temp schema (side-effect, CTE continues) preserve → checkpoint index into CTE chain (pure state, no SQL) restore → truncate chain back to checkpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 6086da0 commit b341018

1 file changed

Lines changed: 25 additions & 7 deletions

File tree

‎docs/PLAN.md‎

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -357,13 +357,31 @@ Based on commands used in [korenmiklos/ceo-value](https://github.com/korenmiklos
357357
- **Test:** local macros from balance.do, foreach from balance.do
358358

359359
### M15: `tempfile`, `preserve`/`restore`
360-
- `preserve` → push current CTE chain + labels onto a stack in state
361-
- `restore` → pop the stack, restoring previous chain + labels
362-
- `tempfile name` → register a named temporary storage slot
363-
- `save "\`name'", replace` → store current chain in the temp slot (as a CTE snapshot)
364-
- `use "\`name'", clear` → restore chain from the temp slot
365-
- State changes: add `vector<ChainSnapshot> preserve_stack` and `map<string, ChainSnapshot> tempfiles` to `StataDoStateInfo`
366-
- **Test:** preserve/restore from manager-facts.do, tempfile from edgelist.do
360+
361+
Stata uses `tempfile` as a workaround for its single-dataset model. In DuckDB we can do better: tempfiles become real tables in a private schema, and preserve/restore becomes a checkpoint in the CTE chain.
362+
363+
**`tempfile`:**
364+
- `tempfile name` → registers `name` as a tempfile identifier in state
365+
- `save "\`name'", replace` → detects tempfile target, executes `CREATE OR REPLACE TABLE _stata_temp.name AS (full CTE chain SELECT)`. This is a side-effect — the CTE chain continues unchanged after save.
366+
- `use "\`name'", clear` → detects tempfile source, resets chain with `_s0 AS (SELECT * FROM _stata_temp.name)`
367+
- Use a dedicated schema `_stata_temp` (created on first use via `CREATE SCHEMA IF NOT EXISTS _stata_temp`) to avoid clashing with user tables
368+
- State: `set<string> tempfile_names` to track which names are tempfiles
369+
370+
**`preserve`/`restore`:**
371+
- `preserve` → snapshot the current CTE chain state: save `step_counter` and `cte_steps.size()` as a checkpoint index into the state
372+
- `restore` → truncate `cte_steps` back to the checkpoint and reset `step_counter`, effectively rewinding to the preserve point
373+
- No tables created, no SQL executed — pure state manipulation
374+
- State: `int preserve_checkpoint = -1` (index into cte_steps, -1 = no active preserve)
375+
- Nested preserve not supported (Stata doesn't support it either)
376+
377+
**Commands:**
378+
- `tempfile` → transformation (registers name)
379+
- `preserve` → transformation (saves checkpoint)
380+
- `restore` → transformation (rewinds chain)
381+
382+
**Test:**
383+
- tempfile from edgelist.do: `tempfile firms; save "\`firms'"; ... use "\`firms'"`
384+
- preserve/restore from manager-facts.do: `preserve; ...; save ...; restore`
367385

368386
### M16: `xtset` + lag/lead operators (`L.`, `F.`)
369387
- `xtset panelvar timevar` → store panel structure in state

0 commit comments

Comments
 (0)