Add OG-Core results visualization workspace - #525
Conversation
Cancelling a queued run only dropped it from the queue, so it stayed pending forever: nothing revisits a pending run, only a running one is repaired on a status read. It now gets the same terminal state a cancelled active run gets. Result tables named the base slot "Baseline" whichever run was in it, so viewing a reform on its own called it a baseline. OG-Core labels the two slots by position and takes no label argument, so a one-run table is relabelled with the run name on our side. Comparisons still read baseline/reform. Applies to the macro, inequality, gini and time series tables, and to the CSV download. Also stops an unwritable run_meta from breaking the queue drain and stranding everything behind it, and adds pytest coverage for the queue, cancel, labelling, run guards and restart recovery.
The parameter form came back empty for every country calibration: 129 of 133 parameters had no title, description, default or range. A country defaults file lists plain values while the base file carries the metadata, and the overlay projected both the same way, so it replaced everything it touched. The overlay now keeps the base metadata and swaps in the country's own value, which is also the value the user actually has. Deleting a baseline removes the whole case, but unlike deleteCase it asked for no session, so it was an unguarded case delete. It now clears the same gate; deleting a reform is unchanged. Parameters could also be changed while a run was already running or queued. The worker reads them when it launches, so that either left a finished run's saved parameters disagreeing with its results, or let a queued run slip past the dimension check it had already passed. saveParams and uploadTaxParams now refuse until the run finishes. Also: reforms record their baseline by name and resolve the path at launch, so a case restored on another machine still finds it; run_meta is written atomically since the status endpoints poll it, and one unreadable meta no longer empties the run list; restoreCase caps the upload and what it may expand into; run ids are no longer reused after a delete; and the reform dimension check now includes consumption goods alongside S, T, J and M.
# Conflicts: # API/app.py
Installing or updating a calibration rewrites the venv a solve is running under, so the two now refuse to overlap: a run will not start while an install is in flight for that country, and installing, updating or re-registering a calibration is refused while a run is using it. A first install is unaffected, since a country with nothing installed cannot have a run. The worker was spawned without -u. Writing to a pipe, Python block-buffers stdout, so a solve's progress sat in the child until it exited: an hour-long run showed one log line and never reported an iteration. It is spawned unbuffered now. A run left behind by a crash is repaired at startup as well as on the next status read, and its worker is killed if it outlived the server. The kill only fires when the recorded pid is still that run's worker, so a reused pid is never touched, and a run directory no longer matches one it is a prefix of. Stopping the server stops a running solve alongside a running install. Also: getRunStatus reports why a run failed rather than only that it did; a failed status read no longer overwrites a run that finished in the meantime; cancel kills outside the lock so it cannot block status polls; the wealth moments table accepts the data moments OG-Core needs to build it at all; and runs get an optional inactivity ceiling, off by default because a healthy solve can be quiet for a long time.
A malformed MUIOGO_OGC_RUN_TIMEOUT_SECONDS raised at import and took the whole app down, while the inactivity value beside it fell back quietly. Both read through one helper now, so a typo in a tuning knob leaves the default in place. Also covers the repair that only fires after a re-read, so a solve that finishes while its status is being polled is not written off as failed.
Without -ww, ps clips the command to the terminal width, or to $COLUMNS when there is no terminal. The run directory sits at the end of a worker's command line, so it was being cut off and the orphan check could never match: on Linux and macOS a leftover worker would simply never be recognised. It surfaced as a test failure because pytest sets COLUMNS.
OG-Core builds this table from a "Data" column of survey moments, and leaves that column empty when none are given, so the frame cannot be built at all and the endpoint failed for every calibration. Hand it blanks and drop the column afterwards, which leaves the model's own numbers. A scalar broadcasts, so this does not depend on how many moments the table has. Passing real data moments still returns both columns.
OGResults.run_completed and OGRunner.alive were written during the build and never called from anywhere, including the tests. The run layer's own docstring still described a single wall-clock watchdog, from before the inactivity ceiling and the orphan kill were added. The rest is whitespace left by earlier edits, and one copy of a test helper that existed in two files. No behaviour changes.
Functional testing on a deployed backend turned up two small gaps in the analysis tables. time_series took no options through the API while the worker behind it accepts stationarized, so the option could not be reached from outside. Same route/worker mismatch as the macro table output_type and the wealth moments data_moments. Asking for the macro table with only a steady state solved was refused with the path of the pickle it could not find, which tells the caller nothing they can act on. It now says there are no transition path results and to run with the full time path, matching what getResults already said. Only the transition path case changed; a missing steady state or model_params still names the file, since neither is something a caller can fix and the path helps when diagnosing one.
restoreCase unpacked the backup straight into the final case folder. A restore that stopped partway, on a full disk or a name the filesystem refuses, left the files written so far sitting under the case name. That wreckage read as a real case, and it also blocked the obvious fix of simply trying again, because the name now existed. Unpack into a staging directory and publish with one rename, so a failed restore leaves nothing at all. Staging goes beside the cases directory rather than inside it: same filesystem, so publishing stays a rename, and a restore in flight is never visible to the two things that walk the cases directory, list_cases and the startup reconcile pass, both of which treat any directory there as a case. The move can still lose a race, since the earlier existence check is not a lock, so a failure there reports the case as already existing when it now does. Nothing is published in that case either. Tests cover the round trip, the refusal to overwrite, and the three things the old code got wrong: no half-case is published, the retry after a failure works, and nothing appears under cases while unpacking.
Case names were global, so "Baseline" could only exist once across every country. Creating it a second time either hit a confusing refusal about country_id or, if the client left country_id out, silently edited the other country's case. Cases now live at cases/<country_id>/<casename>. A case is identified by the pair, so country_id is required wherever an endpoint names one, and the session carries both halves. The directory a case sits in decides its country: genData is pinned to it on write, and the listing reads it from the path. Two things fall out of that. is_country_running answers from the run's own key instead of reading genData, so an unreadable case cannot hide a live run from the install guard. And an edit can no longer move a case between countries, which retires the country_id immutability check. Cases stored in the old layout are moved under their country at startup. One that records no country is left alone rather than guessed at.
# Conflicts: # API/Classes/OGCore/OGCoreCase.py # API/Classes/OGCore/RunJob.py # API/Routes/OGCore/OGCoreRunRoute.py # tests/ogcore/test_run_queue_cancel.py
67546dc to
e5f9904
Compare
|
@error9098x Well-built page! Lazy-loading ECharts instead of adding a megabyte to Three blocking issues — all cases where the page displays wrong analysis rather than failing loudly — then smaller correctness, performance, and cleanup. Blocking1. The policy-change panel hides real differences between the two runs being compared
// OGResults.js:543
let baseValue = name in OGResults.baseParams ? OGResults.baseParams[name] : defaultValue;
let reformValue = name in OGResults.reformParams ? OGResults.reformParams[name] : baseValue;That is not what the worker does. It's the normal path, not an edge case: Repro:
Proposed fix: make the fallback the calibration default on both sides, matching the worker: let baseValue = name in OGResults.baseParams ? OGResults.baseParams[name] : defaultValue;
let reformValue = name in OGResults.reformParams ? OGResults.reformParams[name] : defaultValue;That turns the hidden divergence into a visible row, which is the honest outcome. Worth flagging to the #522 author separately that the reform not inheriting its baseline's parameters may itself be unintended — but this panel should report what actually ran either way. 2. Level-table column headers show group indices instead of group names// OGResults.js:872
$.each(OGResults.groups, label => headers.push('Baseline · ' + label, 'Reform · ' + label));jQuery's Repro: Results → Explore → variable Proposed fix: 3.
|
3 participants
Summary
What changed:
Why:
Open-source UI libraries
Results visualization
The Results tab uses Apache ECharts as its visualization engine, replacing the existing result-viewer-specific
charting approach with a model-neutral renderer that supports the required economic charts and future extension to other models.
Apache ECharts is open source under the Apache License 2.0. Version 6.1.0 is vendored in the
repository with the applicable license and notice files.
Parameter tables
The Parameters page uses Tabulator for structured, editable parameter tables. It provides the required table editing,
validation, sorting, and navigation behavior without introducing a custom grid implementation.
Tabulator is open source under the MIT License. Version 6.5.0 and its license are vendored in the
repository.
Linked issue (if applicable)
Screenshots
Results overview
Caption: Overview of the selected OG-Core results.
Explore view
Caption: Exploring a selected result by variable and comparison.
Results tables
Caption: OG-Core analysis tables and export controls.
Demo:
2026-08-18.at.11_45.PM-3.mp4
Validation
Checklist