Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
9409cf3
diagram: add flow and view invariant checkers for editing
bpowers Sep 11, 2026
11d577a
diagram: add a pure flow geometry core
bpowers Sep 11, 2026
921fb8c
diagram: commit edits through one engine executor
bpowers Sep 11, 2026
3988a0d
notebook-widget: remount instead of reloading the page
bpowers Sep 11, 2026
b6b6148
engine: normalize imported xmile flow geometry in centers
bpowers Sep 11, 2026
54839e4
engine: bring imported mdl pipes onto the 45x35 stock faces
bpowers Sep 11, 2026
5aae6ac
engine: attach every imported mdl flow end per the model
bpowers Sep 11, 2026
1be1e26
engine: lay out a flow listed by two stocks once
bpowers Sep 11, 2026
58e2c84
engine: store and integrate each stock flow once
bpowers Sep 11, 2026
fc7f53c
engine: keep laid-out pipes off stock corners and on valves
bpowers Sep 11, 2026
42b435b
engine: leave flows a patch does not touch where they are
bpowers Sep 11, 2026
bb98a2f
engine: read each stock's flow set before expansion
bpowers Sep 11, 2026
2a32a03
engine: present an mdl flow by the copy nearest its stocks
bpowers Sep 11, 2026
c852496
engine: hold flow legs and valves to the editor's units
bpowers Sep 11, 2026
5d063c7
engine: give created flows joint slots and separate clouds
bpowers Sep 11, 2026
4d48e44
engine: settle the flow geometry arms mutants left open
bpowers Sep 11, 2026
3eabe58
engine: state flow geometry test properties, not history
bpowers Sep 11, 2026
9682b33
engine: heal flow pipes without folds or new crossings
bpowers Sep 11, 2026
0c1ccb0
engine: size attach steps by the segment they leave
bpowers Sep 11, 2026
8a79199
engine: keep created flow ends a pipe spacing apart
bpowers Sep 11, 2026
af7b686
engine: keep a laid-out valve off its segment's ends
bpowers Sep 11, 2026
77e99db
engine: pin the flow copy score's tie-breaks
bpowers Sep 11, 2026
ea5e4c9
engine: refuse added crossings only where G6 holds
bpowers Sep 11, 2026
48951be
engine: keep a laid-out valve off other flows' pipes
bpowers Sep 11, 2026
443b5ac
engine: pin the joint line's second face and the farthest slot
bpowers Sep 11, 2026
0b41520
diagram: plan every canvas gesture in one pure module
bpowers Sep 12, 2026
c937695
diagram: route canvas gestures through the gesture planner
bpowers Sep 12, 2026
e2639a8
diagram: fix gesture lifecycle gaps found in review
bpowers Sep 12, 2026
7e42e50
diagram: route flows around every stock body
bpowers Sep 12, 2026
67430f7
diagram: pin that an invalidated gesture stays dropped
bpowers Sep 12, 2026
1f25675
diagram: skip model refresh for geometry-only edits
bpowers Sep 12, 2026
606a102
notebook-widget: drive diagram editing gestures in a browser
bpowers Sep 12, 2026
10e37d9
notebook-widget: wait for the fake kernel's accept in e2e
bpowers Sep 12, 2026
a8eb2a5
diagram: fix cancel baseline, advisories, queued creates
bpowers Sep 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
- [design-plans/2026-08-25-compiler-unification.md](design-plans/2026-08-25-compiler-unification.md) -- Engine compiler de-duplication, landed: one fragment compiler (`lower_fragment`/`DepShape`), one `BuiltinFn::signature` table, one temp allocator and materialization pass, one axis matcher (`match_axes`), structured `DepRef`s, one `Variable`, one `Diagnostic`, a `(variable, project)`-keyed parse with AST-carried captures and implicit modules, per-variable lowered memos, Loops That Matter as a consumer of each; every pinned semantic divergence and a per-commit measured ledger
- [design-plans/2026-09-04-link-scores-from-fragments.md](design-plans/2026-09-04-link-scores-from-fragments.md) -- LTM link scores synthesized from the target's compiled fragment instead of generated equation text: the rewrite, its per-link keying, the formula families as typed builders, one `LinkScore` opcode, a native post-pass; the roofline for "LTM always on"
- [design-plans/2026-09-07-ltm-single-mode.md](design-plans/2026-09-07-ltm-single-mode.md) -- Single-mode LTM: score every causal edge, compute every loop score after the run from the recorded series (one pipeline for every surface), Johnson only for the structural preview and ids, pins injected into the pipeline, a reducer as a real loop node, static polarity and the shortest-path fallback deleted, sub-model pathway scores emitted only for instances that can lie on a loop; 8 phases
- [design-plans/2026-09-10-diagram-editing-core.md](design-plans/2026-09-10-diagram-editing-core.md) -- Diagram editing core: one flow-geometry owner (terminals, `route`/`routeEnd`, `offsetSegment`, `heal`), one pure gesture planner whose frame at pointer-up is the commit, one controller executor over committed + pending[] edits with rollback; the G1-G8/M1-M3/E1-E6 invariants the tests assert; 7 phases
- [design-plans/2026-08-26-compiler-unification-phase7-investigation.md](design-plans/2026-08-26-compiler-unification-phase7-investigation.md) -- Companion to the compiler unification plan: every parse-time decision that reads model state, where each moves under a `(variable, project)` parse key, every text/name identity site of synthesized helpers, the salsa keys, the capture runlist contract
- [design-plans/2026-08-17-pysimlin-widget.md](design-plans/2026-08-17-pysimlin-widget.md) -- pysimlin file-backed models + anywidget in-notebook editor (file on disk as sync authority)
- [design-plans/2026-04-05-server-rewrite.md](design-plans/2026-04-05-server-rewrite.md) -- Local-first `simlin-serve` binary: filesystem-backed editor + in-process MCP
Expand Down
564 changes: 564 additions & 0 deletions docs/design-plans/2026-09-10-diagram-editing-core.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/tech-debt.md
Original file line number Diff line number Diff line change
Expand Up @@ -623,9 +623,9 @@ Known debt items consolidated from CLAUDE.md files and codebase analysis. Each e

### 67. Connector-error recompute fires on viewport-only project rebuilds

- **Component**: diagram (`src/diagram/project-controller.ts` `attachConnectorErrors` / `updateVariableErrors`, called from `updateProject`)
- **Component**: diagram (`src/diagram/project-controller.ts`)
- **Severity**: low
- **Description**: A sketch-connector/equation consistency check (`ProjectController.attachConnectorErrors`) runs from `updateVariableErrors`, which executes on EVERY project rebuild in `updateProject`. It fetches per-variable equation dependencies via the engine's `Model.getIncomingLinks(varName)` -- one async engine call per on-view aux/flow/stock variable (batched with `Promise.all`). Because viewport-only settles (pan/zoom) route through `queueViewUpdate` -> `updateProject` -> `updateVariableErrors`, this recompute also fires when connectors cannot possibly have changed. For large models (e.g. World3, hundreds of variables) on the Web Worker backend that is N `postMessage` round-trips per pan/zoom settle, adding avoidable latency to the settle's save. The feature is correct as-is; this is purely an efficiency refinement.
- **Description**: (**Resolved** by the controller's executor model: a viewport persist refreshes no annotations, connector dependencies are fetched by a coalesced `connectors` maintenance item requested only after landed edits, the open, undo/redo and navigation, and drift is computed at render time from the cached dependencies. Kept as a historical pointer.) A sketch-connector/equation consistency check (`ProjectController.attachConnectorErrors`) runs from `updateVariableErrors`, which executes on EVERY project rebuild in `updateProject`. It fetches per-variable equation dependencies via the engine's `Model.getIncomingLinks(varName)` -- one async engine call per on-view aux/flow/stock variable (batched with `Promise.all`). Because viewport-only settles (pan/zoom) route through `queueViewUpdate` -> `updateProject` -> `updateVariableErrors`, this recompute also fires when connectors cannot possibly have changed. For large models (e.g. World3, hundreds of variables) on the Web Worker backend that is N `postMessage` round-trips per pan/zoom settle, adding avoidable latency to the settle's save. The feature is correct as-is; this is purely an efficiency refinement.
- **Suggested fix**: (a) Skip the connector recompute on non-content updates by distinguishing viewport-only rebuilds from content/structure edits, or (b) memoize per-variable incoming-links keyed on the serialized model content so pan/zoom settles reuse the prior result.
- **Owner**: unassigned
- **Discovered**: sketch-connector/equation consistency-check work on branch `editor-bug-burndown-2026-07`
Expand Down
1 change: 1 addition & 0 deletions src/core/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ For build/test/lint commands, see [docs/dev/commands.md](/docs/dev/commands.md).
- Variables carry BOTH `ident` (canonical; the `Model.variables` Map key, and it must match engine-canonical idents in sim results, error details, and view-element lookups) and an optional `rawName` (the display spelling from the wire `name`, e.g. "Total Students"). `*FromJson` populates `rawName`; `*ToJson` emits `rawName ?? ident` as `name`. This is what keeps the editor's full-upsert paths from downgrading an imported model's display names one edit at a time (issue #906): the engine stores the payload's `name` verbatim and does all matching canonically (issue #890), so the payload's spelling is authoritative for presentation.
- `Stock`, `Flow`, and `Aux` interfaces all carry `canBeModuleInput` and `isPublic` boolean fields. These are read from `compat` in JSON deserialization and written back to `compat` when true. The fields control which variables appear as module input/output ports in the diagram editor.
- The full engine `compat` field set round-trips through `datamodel.ts`: `activeInitial`, `nonNegative`, `canBeModuleInput`, `isPublic`, `dataSource`, plus the conveyor/queue markers -- `conveyor`/`queue` on `Stock`, `leakage`/`spreadflow`/`overflow` on `Flow`. This is load-bearing: the editor re-serializes a variable as a FULL upsert on any edit (`Editor.tsx` via `stockToJson`/`flowToJson`/`auxToJson`), so any compat field the conversion drops is silently stripped from the model the moment an unrelated field is edited. `Aux` and `Module` deliberately omit the conveyor markers (the engine's uniform Compat accepts them there, but no importer or editor produces them on those kinds).
- Sim results attach to variables as `data` through `groupSeriesByIdent(data, previous?)` (series grouped per variable ident; a variable whose Series objects are unchanged keeps `previous`'s array) and `projectAttachSeries(project, grouped, modelName)` (attaches those arrays by identity). `projectAttachData` groups afresh on every call. A host that re-renders the project often must attach a stable grouping: the diagram's sparklines memoize on the array's identity.
- `Model.macroSpec?: MacroSpec` (`parameters`/`primaryOutput`/`additionalOutputs`) is set exactly when the model is a callable macro template (imported `:MACRO:` / XMILE `<macro>`). `macroSpecFromJson`/`macroSpecToJson` round-trip it; `additionalOutputs` is omitted from JSON when empty. Consumers gate macro-marked models out of module-reference UI (`@simlin/diagram`'s `isMacroModel`).

## Tests
Expand Down
98 changes: 79 additions & 19 deletions src/core/datamodel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,16 @@ export interface SimError {
readonly details: string | undefined;
}

// An engine advisory about a variable that is not a unit error: a
// Warning-severity diagnostic (for example a stock whose flow list repeats a
// flow). The variable still simulates, so a warning never stands in for its
// results. `details` is the engine's reason; the code is often the wire
// `Generic`, which says nothing on its own.
export interface VariableWarning {
readonly code: ErrorCode;
readonly details: string | undefined;
}

// A sketch-hygiene issue: the sketch connectors for a variable have drifted out
// of sync with its equation. `missingConnector` -- the equation references
// `ident` but no connector is drawn from it; `staleConnector` -- a connector is
Expand Down Expand Up @@ -592,6 +602,9 @@ export interface Stock {
// Optional so the many Variable literals that predate this feature stay valid;
// absent and undefined are equivalent ("no connector issues").
readonly connectorErrors?: readonly ConnectorError[] | undefined;
// Engine advisories that are not unit errors, attached by the diagram layer.
// Optional like connectorErrors; absent and undefined are equivalent.
readonly warnings?: readonly VariableWarning[] | undefined;
readonly uid: number | undefined;
}

Expand Down Expand Up @@ -626,6 +639,9 @@ export interface Flow {
// Optional so the many Variable literals that predate this feature stay valid;
// absent and undefined are equivalent ("no connector issues").
readonly connectorErrors?: readonly ConnectorError[] | undefined;
// Engine advisories that are not unit errors, attached by the diagram layer.
// Optional like connectorErrors; absent and undefined are equivalent.
readonly warnings?: readonly VariableWarning[] | undefined;
readonly uid: number | undefined;
}

Expand All @@ -652,6 +668,9 @@ export interface Aux {
// Optional so the many Variable literals that predate this feature stay valid;
// absent and undefined are equivalent ("no connector issues").
readonly connectorErrors?: readonly ConnectorError[] | undefined;
// Engine advisories that are not unit errors, attached by the diagram layer.
// Optional like connectorErrors; absent and undefined are equivalent.
readonly warnings?: readonly VariableWarning[] | undefined;
readonly uid: number | undefined;
}

Expand Down Expand Up @@ -691,6 +710,9 @@ export interface Module {
// Optional so the many Variable literals that predate this feature stay valid;
// absent and undefined are equivalent ("no connector issues").
readonly connectorErrors?: readonly ConnectorError[] | undefined;
// Engine advisories that are not unit errors, attached by the diagram layer.
// Optional like connectorErrors; absent and undefined are equivalent.
readonly warnings?: readonly VariableWarning[] | undefined;
readonly uid: number | undefined;
}

Expand All @@ -702,11 +724,13 @@ export function variableIsArrayed(v: Variable): boolean {
}

export function variableHasError(v: Variable): boolean {
// Includes non-fatal warnings (unit errors, sketch-connector drift), matching
// how the diagram surfaces every variable problem with the same indicator.
// Simulatability is decided separately (engine.isSimulatable), so a
// connector-only warning never blocks a run.
return v.errors !== undefined || v.unitErrors !== undefined || v.connectorErrors !== undefined;
// Includes non-fatal warnings (unit errors, sketch-connector drift, engine
// advisories), matching how the diagram surfaces every variable problem with
// the same indicator. Simulatability is decided separately
// (engine.isSimulatable), so a warning never blocks a run.
return (
v.errors !== undefined || v.unitErrors !== undefined || v.connectorErrors !== undefined || v.warnings !== undefined
);
}

export function variableGf(v: Variable): GraphicalFunction | undefined {
Expand Down Expand Up @@ -1240,7 +1264,11 @@ export function auxViewElementFromJson(json: JsonAuxiliaryViewElement, auxVar?:
var: auxVar?.type === 'aux' ? auxVar : undefined,
x: json.x,
y: json.y,
labelSide: (json.labelSide ?? 'right') as LabelSide,
// An absent side is Center, as the engine reads it (json.rs
// label_side_from_string) and as every other element kind here defaults:
// any other default rewrites the stored side the first time a view is
// upserted, and draws a label the Rust renderer places at the center.
labelSide: (json.labelSide ?? 'center') as LabelSide,
isZeroRadius: false,
};
}
Expand Down Expand Up @@ -2086,29 +2114,61 @@ export function projectToJson(project: Project): JsonProject {
}

export function projectAttachData(project: Project, data: ReadonlyMap<string, Series>, modelName: string): Project {
const model = defined(project.models.get(modelName));
return projectAttachSeries(project, groupSeriesByIdent(data), modelName);
}

// Group every result series by its base variable ident. A scalar variable's
// series is keyed by the bare canonical ident; an arrayed variable's
// per-element series are keyed `ident[<canonical subscripts>]` for any
// dimensionality (1-D `x[a]`, multi-D `x[a,b]`). Grouping by the ident before
// the first `[` attaches every element series -- so multi-dimensional
// variables are plotted too -- and matches whatever the simulation emitted
// rather than reconstructing keys from a Dimension's (original-case)
// subscripts, which avoids the element-name canonicalization mismatch
// entirely.
const seriesByIdent = new Map<string, Series[]>();
/**
* Simulation result series grouped by the variable they belong to. A scalar
* variable's series is keyed by the bare canonical ident; an arrayed variable's
* per-element series are keyed `ident[<canonical subscripts>]` for any
* dimensionality (1-D `x[a]`, multi-D `x[a,b]`) and grouped, in result order,
* under the ident before the first `[`. That attaches every element series --
* so multi-dimensional variables are plotted too -- and matches whatever the
* simulation emitted rather than reconstructing keys from a Dimension's
* (original-case) subscripts, which avoids the element-name canonicalization
* mismatch entirely.
*
* `previous` is the grouping of an earlier result: a variable whose series are
* the same objects in the same order keeps that grouping's array. The array
* becomes the variable's `data`, and the diagram's sparklines memoize on its
* identity, so a variable whose results did not change must keep its array.
*/
export function groupSeriesByIdent(
data: ReadonlyMap<string, Series>,
previous: ReadonlyMap<string, readonly Series[]> = new Map(),
): ReadonlyMap<string, readonly Series[]> {
const grouped = new Map<string, Series[]>();
for (const [key, s] of data) {
const open = key.indexOf('[');
const ident = open === -1 ? key : key.slice(0, open);
const existing = seriesByIdent.get(ident);
const existing = grouped.get(ident);
if (existing) {
existing.push(s);
} else {
seriesByIdent.set(ident, [s]);
grouped.set(ident, [s]);
}
}
const seriesByIdent = new Map<string, readonly Series[]>();
for (const [ident, series] of grouped) {
const before = previous.get(ident);
const unchanged =
before !== undefined && before.length === series.length && before.every((s, i) => s === series[i]);
seriesByIdent.set(ident, unchanged ? before : series);
}
return seriesByIdent;
}

/**
* `modelName`'s variables with their grouped result series (see
* `groupSeriesByIdent`) attached as `data`, by identity. A variable with no
* series is left unchanged.
*/
export function projectAttachSeries(
project: Project,
seriesByIdent: ReadonlyMap<string, readonly Series[]>,
modelName: string,
): Project {
const model = defined(project.models.get(modelName));
const variables = mapValues(model.variables, (v: Variable) => {
const series = seriesByIdent.get(v.ident);
if (!series || series.length === 0) {
Expand Down
113 changes: 113 additions & 0 deletions src/core/tests/datamodel.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ import {
projectFromJson,
projectToJson,
projectAttachData,
projectAttachSeries,
groupSeriesByIdent,
variableHasError,
ErrorCode,
} from '../datamodel';
Expand Down Expand Up @@ -1944,6 +1946,117 @@ describe('projectAttachData', () => {

expect(v?.data).toBeUndefined();
});

it('projectAttachSeries attaches the grouped arrays by identity', () => {
const project = projectWith(
[arrayedAux('population', ['region'])],
[{ name: 'region', subscripts: ['boston', 'nyc'] }],
);
const grouped = groupSeriesByIdent(
new Map<string, Series>([
['population[boston]', series('population[boston]', [10, 11])],
['population[nyc]', series('population[nyc]', [20, 21])],
]),
);
const first = projectAttachSeries(project, grouped, 'main');
const second = projectAttachSeries(project, grouped, 'main');
const data = (p: Project) => defined(p.models.get('main')).variables.get('population')?.data;
expect(data(first)).toBe(grouped.get('population'));
// Attaching the same grouping again (a re-render) hands out the same array.
expect(data(second)).toBe(data(first));
});
});

describe('groupSeriesByIdent', () => {
const series = (name: string, values: number[]): Series => ({
name,
time: new Float64Array([0, 1]),
values: new Float64Array(values),
});
const a = series('a', [1, 2]);
const x1 = series('x[one]', [3, 4]);
const x2 = series('x[two]', [5, 6]);

it('groups scalar and per-element series under the variable ident, in result order', () => {
const grouped = groupSeriesByIdent(
new Map([
['x[two]', x2],
['a', a],
['x[one]', x1],
]),
);
expect([...grouped.keys()].sort()).toEqual(['a', 'x']);
expect(grouped.get('a')).toEqual([a]);
expect(grouped.get('x')).toEqual([x2, x1]);
});

// Every arm of the reuse decision: a variable keeps the previous array only
// when its series are the same objects, the same number, in the same order.
const previous = groupSeriesByIdent(
new Map([
['a', a],
['x[one]', x1],
['x[two]', x2],
]),
);
const rows: ReadonlyArray<{ name: string; data: Array<[string, Series]>; keepsX: boolean }> = [
{
name: 'the same series objects',
data: [
['a', a],
['x[one]', x1],
['x[two]', x2],
],
keepsX: true,
},
{
name: 'an equal but new series object',
data: [
['a', a],
['x[one]', series('x[one]', [3, 4])],
['x[two]', x2],
],
keepsX: false,
},
{
name: 'a series added',
data: [
['a', a],
['x[one]', x1],
['x[two]', x2],
['x[three]', a],
],
keepsX: false,
},
{
name: 'a series removed',
data: [
['a', a],
['x[one]', x1],
],
keepsX: false,
},
{
name: 'the series reordered',
data: [
['a', a],
['x[two]', x2],
['x[one]', x1],
],
keepsX: false,
},
];
for (const row of rows) {
it(`${row.name}: ${row.keepsX ? 'keeps' : 'replaces'} the variable's array`, () => {
const next = groupSeriesByIdent(new Map(row.data), previous);
expect(next.get('a')).toBe(previous.get('a'));
if (row.keepsX) {
expect(next.get('x')).toBe(previous.get('x'));
} else {
expect(next.get('x')).not.toBe(previous.get('x'));
}
});
}
});

// These tests pin the wire fields that an editor upsert (a full variable
Expand Down
Loading
Loading