diff --git a/CHANGELOG.md b/CHANGELOG.md index a7d9666..1867c30 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,8 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **`gemdb` is on the PATH of terminals you open in VS Code.** The command is + generated into `~/GemDB/bin`, which is on nobody's PATH, so `which gemdb` + answered "not found" and the README's answer was a line you had to add to + your own shell profile. GemDB now contributes that directory to the + terminals this editor launches — and takes it back when the extension is + disabled or uninstalled. Your shell profile is still yours; add it there + too if you want `gemdb` in terminals outside VS Code. + +### Changed + +- **The sample code now starts with `import gemdb`.** A new notebook's first + cell, the walkthrough and the README all opened with `import gemstone` — + Grail's own lower-level surface, inherited from Jasper. The first thing a + developer meets is now `gemdb`: `gemdb.root` for the data that outlives the + session, `gemdb.commit()` for the moment it becomes everyone's. + ### Fixed +- **Non-ASCII text now survives `gemdb file.py` in both directions.** + `print()` from a script wrote its characters as UTF-16 code units — a NUL + between every ASCII letter, and anything above U+00FF truncated to one byte + — so a `•` in an ASCII-art rabbit turned the whole drawing into binary. + `input()` had the mirror-image fault: a line typed at the terminal arrived + one character per *byte*, so `wörld` came back six characters long and + mojibake. Only this mode was affected; the GemDB Shell and notebooks were + always right, because they exchange characters over the client connection + rather than bytes through a file. Needs the matching Grail change + (GemTalk/Grail, console writes encode for a byte sink). +- **`gemdb file.py` no longer prints topaz's own commentary.** Running a + script from a real terminal ended with four lines about topaz ignoring an + EXIT command, followed by `Logging out session 1.`. The driver ended with + an `exit` that topaz documents as ignored for the way GemDB runs it — it + already exits when the script completes — and ignoring it is silent on a + pipe but spoken aloud on a terminal, which is why it never showed in CI. + Exit codes are unaffected: they never travelled through that line. - **The GemDB Shell no longer prints the database client's own chatter.** Opening a shell wrote a line like `gcits login: session 0x… lgc 0x… rpc gem processId 4726` onto your terminal diff --git a/CLAUDE.md b/CLAUDE.md index 1dda2fa..e6a35f0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,6 +56,17 @@ reversible; ask about what is persistent or global.** unraised it stands down and leaves that to `ensureRunning`, where a `sudo` dialog has a visible cause. +- Putting `gemdb` on the PATH of terminals VS Code opens — + `putCliOnPath` in `cli.ts`, applied to + `context.environmentVariableCollection`. Automated because VS Code owns the + reversal: the entry applies only to terminals this editor launches and goes + away when the extension is disabled. **Editing the user's shell profile + would be the other side of the line** — persistent, global, not ours to + undo — so the README asks rather than does. `clear()` before every + `prepend` because the collection is persisted across window reloads and + re-applied before activation; without it a reload stacks a second entry and + a changed root path leaves the old one in front. + `ensureRunning` in `lifecycle.ts` is the single path to a running database, whether the user pressed Start or just ran a notebook cell. It finishes any outstanding preparation, prompts for shared memory, starts the processes, and @@ -218,7 +229,13 @@ re-running `bundle:grail` on every supported platform. **The CLI's exit codes go through a status file, not topaz.** topaz cannot carry an exit status out of a `run` block — `ExitClientError status:` is not -translated, and an `iferr … exit 1` action exits 0 (all measured). So +translated, and an `iferr … exit 1` action exits 0 (all measured). The driver +therefore ends with no `exit` command at all, and must not grow one: `topaz -h` +says of `-S` that topaz "exits when the script completes" and that "exit and +quit commands are ignored". Ignored silently when stdin is a pipe — which is +every CI run, every test, and every `gemdb x.py | cat` — and out loud when +stdin is a tty, where it printed four lines of explanation and a +`Logging out session 1.` in front of the user. So `gemdb-run.tpz` writes the status to the file named in `GEMDB_STATUS_FILE` and the bash wrapper becomes the exit code. Errors there are caught as `AbstractException`, not `Error`: Grail's Python exceptions live outside the @@ -294,6 +311,23 @@ executeAsync loop clears the stack at the next forwarder stop and throws `Error: KeyboardInterrupt - `. Anything new that evaluates Python should go through that layer, not `execute` directly. +**The console box says what the sink takes, because the sink cannot be +asked.** `SessionTemps #GrailConsole` holds an Array; slot 1 is the sink, and +slot 2 — `#'utf8'` — declares that it takes bytes. `gemdb-run.tpz` sets it, +because its sink is `GsFile stdout` and `nextPutAll:` writes a Unicode string's +code units straight through: `print('café')` was UTF-16BE on the terminal, a +NUL between every ASCII character, while the same print through the shell was +right. Grail encodes with `nextPutAsUtf8:` when the slot says so. It cannot +instead probe the sink: the shell installs a `ClientForwarder`, and *any* send +to one — `class`, `respondsTo:`, `isNil` — forwards to the client as error +2336, which is not a Smalltalk exception and is not catchable in the gem +(measured; `on: AbstractException` around `forwarder class` does not run). A +probe would turn every print in a streaming session into a spurious client +stop. The read side is the same seam from the other direction: `GsFile stdin` +answers bytes, so Grail decodes that one branch with `decodeFromUTF8`, keeping +the raw line when it is not UTF-8. Both directions are pinned in +`src/__integration__/cli.test.ts`. + **Stage Grail before stamping it, and stamp only what this run created.** `stageAndRecordGrail` in `grail.ts` owns that order. Reversed, it broke both ways at once: on a first install `/grail` does not exist yet, so diff --git a/README.md b/README.md index b753541..71dfaff 100644 --- a/README.md +++ b/README.md @@ -13,10 +13,10 @@ whose objects *are* the database's objects. Assign to a variable, commit, and it is still there tomorrow. ```python -import gemstone +import gemdb -gemstone["routes"] = load_routes() -gemstone.system.commit() +gemdb.root["routes"] = load_routes() +gemdb.commit() ``` ## Getting started @@ -63,14 +63,21 @@ Setup also writes a shell command to `~/GemDB/bin/gemdb` that behaves like CPython's command line, backed by the database: ```sh -export PATH="$HOME/GemDB/bin:$PATH" # once, in your shell profile - gemdb hello.py # like python3 hello.py gemdb -m some.module # like python3 -m gemdb -c 'print(1+1)' # like python3 -c gemdb # the GemDB Shell ``` +Terminals you open in VS Code already have it: GemDB adds `~/GemDB/bin` to +their PATH, and removes it again if you disable the extension. Terminals +outside VS Code are yours, so GemDB does not edit your shell profile — add it +there yourself if you want `gemdb` everywhere: + +```sh +export PATH="$HOME/GemDB/bin:$PATH" # once, in your shell profile +``` + It needs no environment set up — the wrapper carries its own — and if the database is not running it starts it, the same judgement the editor makes. Exit codes work the way scripts expect: 0 on success, 1 on an uncaught diff --git a/docs/demo-rabbit-in-the-hat.md b/docs/demo-rabbit-in-the-hat.md index 233d858..e71a7bb 100644 --- a/docs/demo-rabbit-in-the-hat.md +++ b/docs/demo-rabbit-in-the-hat.md @@ -8,10 +8,11 @@ below was run against a real database on 2026-08-23; where something surprised me, that is noted rather than tidied away. > **Before you start.** GemDB must be installed (open VS Code once and let it -> finish setting up), and `gemdb` must be on your PATH: +> finish setting up). A terminal opened in VS Code has `gemdb` on its PATH +> already; in any other terminal, put it there: > > ```sh -> export PATH="$HOME/GemDB/bin:$PATH" +> export PATH="$HOME/GemDB/bin:$PATH" # not needed in a VS Code terminal > cd docs/demo > ``` > diff --git a/resources/walkthrough/notebook.md b/resources/walkthrough/notebook.md index 37b4657..c7dd229 100644 --- a/resources/walkthrough/notebook.md +++ b/resources/walkthrough/notebook.md @@ -4,8 +4,8 @@ Open a notebook and pick **GemDB** as the kernel. Cells run inside the database and share variables the way you would expect: ```python -import gemstone -routes = gemstone["routes"] # data already in the database +import gemdb +routes = gemdb.root["routes"] # data already in the database len(routes) ``` diff --git a/resources/walkthrough/repl.md b/resources/walkthrough/repl.md index e3fa7a5..4c1d6df 100644 --- a/resources/walkthrough/repl.md +++ b/resources/walkthrough/repl.md @@ -3,10 +3,10 @@ Opens a Python prompt that runs *inside* the database: ``` ->>> import gemstone ->>> gemstone["answer"] = 42 ->>> gemstone.system.commit() ->>> gemstone["answer"] +>>> import gemdb +>>> gemdb.root["answer"] = 42 +>>> gemdb.commit() +>>> gemdb.root["answer"] 42 ``` diff --git a/src/__integration__/cli.test.ts b/src/__integration__/cli.test.ts index 4c35b08..8fbbabb 100644 --- a/src/__integration__/cli.test.ts +++ b/src/__integration__/cli.test.ts @@ -145,6 +145,28 @@ describe.skipIf(!ready || !canMakeFixture())('the gemdb command', () => { expect(ran.code).toBe(0); }); + it('writes non-ASCII output as UTF-8, not UTF-16 code units', async () => { + // The console this mode installs is a GsFile, which takes BYTES: writing + // a Unicode string to it straight put a NUL between every ASCII character + // and truncated anything above U+00FF to its low byte, so an ASCII-art + // rabbit with a bullet in it came out as binary. Grail encodes because + // the driver's #GrailConsole box declares the sink takes utf8. + const ran = await gemdb('-c', 'print("caf\u00e9 \u2022 \u4e2d")'); + expect(ran.stdout).toContain('caf\u00e9 \u2022 \u4e2d'); + expect(ran.stdout).not.toContain('\u0000'); + expect(ran.code).toBe(0); + }); + + it('reads non-ASCII input as characters, not bytes', async () => { + // The other half of the same seam: GsFile stdin answers bytes, so a + // terminal's UTF-8 arrived one Character per byte — 'w\u00f6rld' as + // 'w\u00c3\u00b6rld', six characters long, and mojibake on the way back + // out. + const ran = await run(['-c', 'n = input()\nprint(n, len(n))'], 'w\u00f6rld\n'); + expect(ran.stdout).toContain('w\u00f6rld 5'); + expect(ran.code).toBe(0); + }); + it('raises EOFError when stdin runs dry', async () => { const ran = await run(['-c', 'input()'], ''); expect(ran.stderr).toContain('EOF'); diff --git a/src/__tests__/cli.test.ts b/src/__tests__/cli.test.ts index d054655..f74e6a7 100644 --- a/src/__tests__/cli.test.ts +++ b/src/__tests__/cli.test.ts @@ -3,7 +3,7 @@ import * as os from 'os'; import * as path from 'path'; import { afterEach, beforeEach, describe, expect, it } from 'vitest'; import { __setSetting } from '../__mocks__/vscode'; -import { cliPath, ensureCliCurrent, writeCliScripts } from '../cli'; +import { cliPath, ensureCliCurrent, putCliOnPath, writeCliScripts } from '../cli'; import { cliStampPath, expectedEnginePath } from '../paths'; /** @@ -78,6 +78,16 @@ describe('writeCliScripts', () => { // A file run is linked topaz, so it never reaches session.ts and would // otherwise sit in the shared cache as the stock 'TopazL'. The truncation // is not optional: 32 characters raises OutOfRange, at login. + // No `exit` command: with -S, topaz exits when the script completes and + // ignores exit outright — quietly on a pipe, but on a tty it prints four + // lines about ignoring it and a spurious "Logging out session 1.". A test + // here because a pipe is what both suites use, so nothing else would + // notice it coming back. + expect(run.split('\n').some((line) => /^\s*(exit|quit)\b/.test(line))).toBe(false); + // The console sink is a GsFile, which takes bytes; the second slot of + // the #GrailConsole box is what tells Grail to encode. Without it, + // non-ASCII print() output is UTF-16 code units on the terminal. + expect(run).toContain("put: (Array with: GsFile stdout with: #'utf8')"); expect(run).toContain('System cacheName: label'); expect(run).toContain("label := 'GemDB run ', label"); expect(run).toContain('label size > 31 ifTrue:'); @@ -199,3 +209,61 @@ describe('writeCliScripts', () => { expect(() => writeCliScripts(ext)).toThrow(/not installed/); }); }); + +describe('putCliOnPath', () => { + /** A stand-in for the collection VS Code applies to its own terminals. */ + function fakeEnvironment(): { cleared: number; prepended: [string, string][] } & { + clear(): void; + prepend(variable: string, value: string): void; + } { + return { + cleared: 0, + prepended: [], + clear() { + this.cleared += 1; + }, + prepend(variable: string, value: string) { + this.prepended.push([variable, value]); + }, + }; + } + + it('prepends the directory the command is generated into', () => { + const env = fakeEnvironment(); + + putCliOnPath(env); + + // Prepended, not appended: a `gemdb` of GemDB's own is the one this + // installation generated. And the trailing delimiter is the whole + // mechanism — without it the entry runs into the rest of the PATH. + expect(env.prepended).toEqual([['PATH', `${path.join(root, 'bin')}${path.delimiter}`]]); + }); + + it('clears first, because the collection outlives the window', () => { + // VS Code persists it across reloads and re-applies it before activation. + // Re-applying without clearing would stack an entry per reload, and a + // changed root path would leave the old one in front of the new. + const env = fakeEnvironment(); + + putCliOnPath(env); + __setSetting('gemdb.rootPath', path.join(root, 'elsewhere')); + putCliOnPath(env); + + expect(env.cleared).toBe(2); + expect(env.prepended[1]).toEqual([ + 'PATH', + `${path.join(root, 'elsewhere', 'bin')}${path.delimiter}`, + ]); + }); + + it('contributes the directory even before the command is written', () => { + // First run: the PATH is set at activation and setup writes bin/gemdb + // minutes later. A terminal opened in between must not need reopening. + fs.rmSync(path.join(root, 'bin'), { recursive: true, force: true }); + const env = fakeEnvironment(); + + putCliOnPath(env); + + expect(env.prepended).toHaveLength(1); + }); +}); diff --git a/src/cli.ts b/src/cli.ts index 247337b..e2cbee3 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -130,6 +130,39 @@ function cliIsCurrent(fingerprint: string): boolean { } } +/** + * The slice of VS Code's `EnvironmentVariableCollection` that putting `gemdb` + * on the PATH needs, so the decision is testable without an editor. + */ +export interface TerminalEnvironment { + clear(): void; + prepend(variable: string, value: string): void; +} + +/** + * Put `gemdb` on the PATH of terminals VS Code opens. + * + * The command is generated into `/bin`, which is on nobody's PATH, + * so `which gemdb` in a VS Code terminal reported nothing and the README's + * answer was an `export PATH=...` line the user had to add to their own + * profile. VS Code contributes this itself, per extension: it applies to + * terminals this editor launches, is reverted when GemDB is disabled or + * uninstalled, and never touches the user's shell profile — which keeps it on + * the automated side of the line (inert, reversible, and not a change to the + * machine), where editing a profile would not be. + * + * `clear()` first because the collection is persisted across window reloads: + * re-applying on activation is what keeps it right after the root path + * changes, and without the clear it would accumulate a stale entry per + * reload. Contributed whether or not `bin/gemdb` exists yet — on first run it + * does not, and a terminal opened during setup should find the command once + * setup writes it rather than needing to be reopened. + */ +export function putCliOnPath(env: TerminalEnvironment): void { + env.clear(); + env.prepend('PATH', `${cliDirPath()}${path.delimiter}`); +} + /** * Guarantee `/bin/gemdb` exists and matches this build, for callers * about to hand that path to something else. @@ -165,6 +198,17 @@ export function writeCliScripts(extensionPath: string): void { // Grail's Python exceptions live outside the Error branch, which is why // grail.tpz's own file mode exits 0 on a Python error. // + // There is deliberately no `exit` command at the end. topaz's own help says + // it of `-S`: the script is processed with INPUT, "topaz exits when the + // script completes", and "exit and quit commands are ignored". Ignored + // quietly when stdin is a pipe, which is why both suites and every run + // through a non-interactive shell missed it — but out loud when stdin is a + // tty, where `gemdb hide.py` printed four lines of topaz explaining that it + // was ignoring the EXIT, plus a "Logging out session 1." the ignored exit + // provoked. The status still leaves through GEMDB_STATUS_FILE, which never + // depended on that line (measured again without it: sys.exit(3) → 3, an + // uncaught exception → 1, a missing file → 2). + // // sys.exit(n) is decoded here, not upstream: Grail raises its own SystemExit // (never ExitClientError — input()'s except SystemExit and finally blocks // must keep working), and the exit argument survives only in the exception's @@ -196,8 +240,19 @@ warnings) consult SessionTemps #GrailConsole first; that write is transient and leaves System needsCommit untouched. Boxed in an Array because that is the #GrailConsole protocol (see builtins.gs ___console___: SessionTemps at:put: sends to what it stores, which a -ClientForwarder cannot survive; a GsFile could, but one protocol)." -SessionTemps current at: #'GrailConsole' put: (Array with: GsFile stdout). +ClientForwarder cannot survive; a GsFile could, but one protocol). + +The second slot declares what this sink takes: a GsFile takes BYTES, so +every console write must be encoded. Without it Grail wrote a Unicode16's +code units straight to stdout -- a NUL between every ASCII character, and +a non-ASCII character truncated to its low byte -- so print('caf\u00e9') +was UTF-16BE on the terminal while the same print through the shell (a +character stream over GCI) was right. Declared here rather than sniffed +there because the sink cannot be asked: the shell's sink is a +ClientForwarder, and asking one anything -- class, respondsTo:, isNil -- +forwards to the client as error 2336, which is not catchable in the gem." +SessionTemps current at: #'GrailConsole' + put: (Array with: GsFile stdout with: #'utf8'). [ [ target := args at: ofs + 1. @@ -259,7 +314,6 @@ SessionTemps current at: #'GrailConsole' put: (Array with: GsFile stdout). f nextPutAll: status printString; close]. ]. % -exit 0 `; const script = `#!/bin/bash diff --git a/src/extension.ts b/src/extension.ts index c9bb98f..bd92dad 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -12,6 +12,7 @@ import { uninstall, } from './lifecycle'; import { autoStartSuppressed, initAutoStart, suppressAutoStart } from './autoStart'; +import { cliDirPath, putCliOnPath } from './cli'; import { withSetupLock } from './lock'; import { disposeLog, log, showLog } from './log'; import { @@ -176,6 +177,8 @@ export function activate(context: vscode.ExtensionContext): void { ) { // Every notebook's session is bound to the old database too. logoutAll(); + // And the `gemdb` on the PATH of new terminals is the old root's. + if (isSupportedPlatform()) putCliOnPath(context.environmentVariableCollection); status.refresh(); } }), @@ -190,6 +193,15 @@ export function activate(context: vscode.ExtensionContext): void { return; } + // `which gemdb` should answer in a terminal opened here. After the platform + // gate: on a platform GemDB cannot run, there will never be a command to + // find, and pointing the PATH at a directory that stays empty explains + // nothing. + // Named with the real directory, not `~/GemDB/bin`: the root path is a + // setting, and this string is what the terminal shows to explain itself. + context.environmentVariableCollection.description = `Adds the \`gemdb\` command (${cliDirPath()}) to the PATH.`; + putCliOnPath(context.environmentVariableCollection); + status.refresh(); void prepareOnFirstRun(context, extensionPath, () => status.refresh()).then(() => diff --git a/src/notebook.ts b/src/notebook.ts index 1cb4ecb..5c22cdb 100644 --- a/src/notebook.ts +++ b/src/notebook.ts @@ -192,19 +192,22 @@ export async function resetActiveNotebook(): Promise { /** * Open a new notebook with one Python cell, ready to run. * - * The starter cell is not decoration: `import gemstone` is the one thing that + * The starter cell is not decoration: `import gemdb` is the one thing that * makes this different from any other Python notebook, and showing it here is - * cheaper than explaining it. + * cheaper than explaining it. `gemdb` is the module a developer should meet + * first — `gemstone` is Grail's own lower-level surface, and a starter cell + * that opens with it teaches the wrong name. */ export async function newNotebook(): Promise { const starter = [ '# Python here runs inside your GemDB database.', - '# The gemstone module reaches the data stored in it.', - 'import gemstone', + '# Everything reachable from gemdb.root is still there tomorrow.', + 'import gemdb', '', - 'gemstone["greeting"] = "Hello from GemDB!"', - 'gemstone.system.commit()', - 'gemstone["greeting"]', + 'gemdb.root["greeting"] = "Hello from GemDB!"', + 'gemdb.commit()', + '', + 'gemdb.root["greeting"]', ].join('\n'); const cell = new vscode.NotebookCellData(vscode.NotebookCellKind.Code, starter, 'python');