Document standalone Linux releases, secondary WSL2 support and remaining work

Author: hyperpolymath

docs/migration/STATUS.md

@@ -0,0 +1,69 @@
+# Migration and distribution status
+
+Updated 2026-09-18. This document distinguishes implemented work from agreed requirements. Packaging policy is in ../../packaging/; UI execution instructions are in ../../ui/README.md.
+
+## Implemented and published before this update
+
+Commit a4b9f81 contains the first opt-in Stipple/Vue UI slice: isolated Julia environment and manifest, read-only studies/list/detail, refresh and error states, typed study DTOs and validated backend adapter, tests, CI and migration inventories. The legacy React app remains the default and the scientific backend is unchanged.
+
+Evidence at that milestone: 38 local Julia checks and browser fixture tests passed. GitHub's main CI and Stipple contract workflow subsequently passed. The GitHub contract workflow runs 31 contract/URL checks; the extra seven HTTP-adapter checks and browser suite were run locally, not by that workflow. Full scientific-backend integration with the new UI remains pending.
+
+## Agreed requirements, not yet delivered functionality
+
+- Julia-authored Stipple/Vue frontend; all existing features retained, progressively. Small JavaScript adapters are allowed. No new application TypeScript; existing TS can remain until replacement reaches parity.
+- Bun only for JavaScript tooling. No npm, pnpm or Deno commands. The new browser test currently still uses Node instructions and must be converted and verified under Bun.
+- Local single-user operation first; authenticated remote/multiuser deployment later. Do not expose the present local server as if it were multiuser-ready.
+- Standalone offline-first release archives for native Linux x86-64 and ARM64. Guix builds the environment; users do not install Guix. Bundle mise, just and Bun, application runtimes/packages, scientific tools and web assets. Users provide sequencing data and reference databases; a compatible host kernel/CPU and existing browser remain prerequisites.
+- Secondary WSL2 compatibility using the Linux archives. Native Linux is primary; WSL1/native Windows are outside scope. Neither native standalone artifacts nor WSL2 compatibility has yet been validated.
+- Automatic coordinated updates while online and idle. Track latest compatible stable versions as one tested, pinned release; no independent in-place component upgrades. Signed metadata, hashes, transactional switch, health checks and rollback required. Startup must work offline.
+
+## Remaining workstreams
+
+### A. Finish the application migration
+
+1. Integrate the new UI against the actual Oxygen backend with disposable data; verify API contracts beyond synthetic fixtures.
+2. Study create/rename/delete with confirmations, invalid-input handling and recovery; groups, runs, pooled-run navigation and legacy deep-link behaviour.
+3. Jobs and pipeline controls: submit stages/runs, status/progress/logs, cancellation, event feed, reconnect snapshots, bounded subscriptions and cleanup. Preserve R runtime coordination and job/provenance semantics.
+4. Default/study/group/run configuration inheritance and overrides; primer/database editors and downloads; composition/category/filter libraries.
+5. Full results table parity: server pagination/sort/filter/distinct values, column settings/persistence, highlighting/copy, OTU details, saved tables, annotation/contamination edits and exports/reports.
+6. Plotly parity, sizing/export/cosmetics, replacement for React chart editor, and Euler/UpSet visualisations. These and the custom table are major remaining parity risks. A Julia chart wrapper alone does not replace a chart editor.
+7. Browser regression and scientific-result comparisons, default switch, then remove React/TypeScript/Vite/Bun frontend build dependencies that are no longer used. Keep Bun for approved JS adapters/tooling. Retain fallback UI until replacements are accepted.
+
+### B. Build real standalone releases
+
+1. Audit runtime closure and every lazy-download path: both Julia environments/artifacts, R/RCall/Bioconductor, Python tools, Java/FastQC, native libraries, executables, static assets and licenses/source obligations.
+2. Define pinned Guix channels/recipes and mise/just build tasks. Current packaging files are policy only; no working Guix release builder exists yet.
+3. Implement architecture-specific release build/assembly. Do not upgrade the Oxygen HTTP 1 environment merely to share the UI's HTTP 2 environment.
+4. Provide unprivileged launcher, writable state outside immutable releases, browser access and diagnostics. Deliver prebuilt legacy UI while migration is incomplete; never build/install at first launch.
+5. Prove relocatable offline execution with no preinstalled application toolchain, Guix or /gnu/store, including paths with spaces and lazy scientific functionality. Establish supported kernel/CPU baseline and storage/RAM guidance; measure artifact sizes rather than guessing.
+6. Verify both native architectures, then explicit WSL2 tests: Linux-filesystem install, Windows-browser localhost access, scientific tools, shutdown and update behaviour. Publish verified versus unverified environments honestly.
+7. Publish architecture-specific archives, hashes, signed metadata and component/license inventory on GitHub Releases. A source ZIP is not the standalone product.
+
+### C. Implement the coordinated updater
+
+1. Stable-version discovery and candidate lockfile updates through release tooling; test the whole combination. Document held-back incompatible versions.
+2. Establish signing/trust-key provisioning and release metadata format; configure publication secrets securely.
+3. Build asynchronous online checking/staging, architecture selection, signature/hash verification, archive extraction safety, expiry and anti-replay rules.
+4. Add a backend-owned exclusive update/job-submission gate so an idle check cannot race with a new job. Future multiuser installations require installation-wide coordination, not per-tab checks.
+5. Atomic release switch, restart/health probes, interruption/disk-full/corruption handling and rollback; keep current release usable offline. Schema/data migration recovery must be explicit: binary rollback alone cannot undo data changes.
+6. Add maintenance/pause controls for operators and failure-mode tests on both architectures/WSL2.
+
+### D. Add remote hosting and multiuser operation
+
+Authentication and session security; roles and study permissions; job ownership/quotas/scheduling; persistence and recovery across backend restarts; installation-wide R/runtime concurrency strategy; protected file/download routes; TLS/reverse-proxy and WebSocket origin policy; backups, auditability and operator-managed updates. Per-page state isolation in the first UI slice is necessary but does not implement these requirements.
+
+### E. Testing and delivery follow-through
+
+Convert Node-based browser instructions to Bun and test them. Add actual-backend/browser checks to CI; add native ARM64 and WSL2 acceptance environments. Keep scientific tests unchanged unless justified separately. Version the feature-parity checklist and publish known limitations. No final cutover until existing features have replacements and release acceptance gates pass.
+
+## Recommended next sequence
+
+1. Start runtime/dependency audit and Guix feasibility work now; in parallel, complete disposable actual-backend validation of the Stipple slice.
+2. Produce the first real offline x86-64 archive retaining the existing full UI plus the opt-in Stipple UI. Packaging need not wait for complete UI migration.
+3. Establish native ARM64 builds and secondary WSL2 verification.
+4. Continue feature migration; add signed update infrastructure after immutable releases and acceptance tests exist.
+5. Add remote/multiuser capabilities as an explicit later delivery stage, not a networking flag.
+
+## What needs owner input later
+
+No more product clarification is needed to begin the audit or next UI slice. Later requirements: suitable build/test capacity for ARM64 and WSL2; secure GitHub authentication and release-signing configuration; representative non-sensitive scientific fixtures; remote host/identity provider and study-sharing rules. Do not send signing private keys or access tokens in chat. Existing shared access tokens should be revoked.

packaging/README.md

@@ -0,0 +1,71 @@
+# Standalone releases: agreed requirements
+
+Status: **requirements recorded; release builder, signed updater and standalone binaries are not implemented yet.** Do not advertise the current source archive as an offline-ready application.
+
+## Release boundary
+
+Separate Linux x86-64 and ARM64 archives. Download, extract and launch without installing Julia, R, scientific tools, Guix, mise, just or Bun. Bundle mise, just and Bun. No npm/pnpm/Deno commands or new application TypeScript. Users supply their sequencing data and reference databases. Therefore offline startup and tool execution are guaranteed targets, but taxonomy workflows still require user-supplied compatible reference databases.
+
+The ordinary kernel/CPU, sufficient storage/RAM, and a supported browser for the WebUI remain host requirements. Do not bundle Chromium by default: its size is disproportionate to a browser-based application. Establish and publish the Linux kernel baseline through clean-host testing. Native desktop/browser embedding would be a separate release-size decision.
+
+All existing features must remain available during migration. Until Stipple reaches parity, bundle the prebuilt legacy UI alongside the new UI; the runtime must not invoke a frontend build or install dependencies. Data and user configuration must live outside immutable release directories.
+
+## Support priorities
+
+- **Primary:** native Linux x86-64 and ARM64. Native operation drives packaging and release design.
+- **Secondary:** WSL2 on matching x86-64 and ARM64 hosts, reusing the Linux archives. This is an intended support target, not a compatibility claim before testing. WSL1 and native Windows execution are outside scope.
+- Users install WSL2 and a Linux distribution themselves. No host Guix or additional application dependencies should be required inside that distribution.
+- Recommend extraction, runtime state and active scientific work in the WSL Linux filesystem (for example `~/metamanifold`), not `/mnt/c/`. Document Windows-filesystem limitations rather than promising equivalent performance or permissions.
+- Use the Windows browser for the UI. Verify localhost access in the supported WSL networking configuration; do not expose the server publicly as a workaround.
+- Add explicit WSL2 checks for offline extraction/launch, relocatable execution, native tools/RCall, browser connectivity, shutdown/restart and idle update/rollback. ARM64 WSL2 must be tested independently before being advertised as verified.
+- Native releases take priority. Clearly publish per-environment validation status and known WSL limitations; secondary support must not be presented as already tested.
+
+## Guix, mise and just responsibilities
+
+- Guix: pinned channels and package recipes, complete runtime closure, architecture-specific build environments and relocatable pack investigation (`guix pack -RR`). Validate actual relocation and execution without host Guix/root or a preexisting /gnu/store; do not assume this works on every Linux host.
+- mise: release-engineering version discovery/pins and bundled tool selection. It must not execute `install`/`upgrade` on application startup or independently mutate an installed scientific environment.
+- just: reproducible build/test/package/update-check entrypoints, backed by bundled tools in delivered archives.
+- Bun: the only JavaScript package manager/runtime/build tool used by this project. Existing Node browser-test commands need conversion and testing through Bun before satisfying this policy.
+
+The current Julia/Guix combination, binary artifacts, RCall, Bioconductor packages, Java-dependent FastQC, Python tools, native libraries and all scientific executables require a packaging audit. A Guix manifest alone is not a complete offline application. Include licenses/notices and required source offers for distributed components.
+
+## Coordinated update policy
+
+Automatically check for releases while online; do not delay launch or fail offline. Discover stable component updates in release engineering, resolve all lockfiles, build and test complete candidates on both architectures, then publish signed metadata plus hashes. Do not use unpinned `latest` versions inside an installed release.
+
+Target the latest mutually compatible stable versions. If the absolute newest versions cannot coexist or scientific regression tests fail, keep the working release and report the held-back components and reasons. No claim that every upstream latest version can always ship simultaneously.
+
+Runtime updater (to implement):
+1. Validate signed metadata against bundled trust roots, platform, version, expiry and replay/rollback rules.
+2. Download into staging without touching the current release; verify hashes and archive extraction safety.
+3. Coordinate through a backend-owned exclusive update/job-submission gate. All jobs and mutations must be idle; no merely client-side idle check.
+4. Take a configuration/data-schema-compatible recovery point before migrations. A binary rollback cannot undo arbitrary data migrations.
+5. Switch the complete release atomically, restart services, perform health checks, and roll back on failure. Retain the previous working release.
+6. On interruption, insufficient disk, invalid signature, offline operation or failure, leave the current installation usable. Provide administrator pause/maintenance controls for future remote hosting.
+
+Reference databases are not silently downloaded or updated. Their versions and checksums belong to scientific provenance, even though their bytes are user supplied.
+
+## Required acceptance gates
+
+- Both architectures boot from complete release archives on clean compatible hosts with networking disabled, no host /gnu/store and no preinstalled application toolchains.
+- Start both UI/backend services; load browser assets with external networking blocked.
+- Run tool probes and an appropriate synthetic scientific smoke test. Database-dependent tests use explicit external test fixtures, not undeclared runtime downloads.
+- Exercise RCall and lazily loaded Julia/native artifacts; no package-manager/network fallback.
+- Verify relocation, spaces in paths, writable state separation, unprivileged launch and documented kernel requirements.
+- Verify new-job/update race handling, interrupted download, corrupted archive, invalid/expired signature, failed health check and rollback.
+- Produce a component/version/license inventory and measured compressed/unpacked sizes. Remove build-only compilers/caches where safe; do not remove scientific capability for size.
+
+## Next implementation work
+
+1. Preserve the published migration baseline while adding packaging (checkout reconciliation completed before this policy publication).
+2. Inventory every runtime dependency and lazy-download path; assess Guix recipes and ARM64 support.
+3. Pin Guix channels and add package recipes plus build-only mise/just definitions, without inventing unverified version hashes.
+4. Build one offline x86-64 closure, then the ARM64 equivalent; prove clean-host execution.
+5. Implement authenticated release publication/updater and failure tests.
+6. Integrate feature-parity UI migration and publish architecture-specific release artifacts.
+
+The declarative release-policy.toml records decisions only. It does not enforce these guarantees yet.
+
+## Overall project status
+
+See [migration and distribution status](../docs/migration/STATUS.md) for implemented work, remaining scope, acceptance gates and later owner inputs.

packaging/release-policy.toml

@@ -0,0 +1,60 @@
+# Release requirements, not an implemented builder or updater.
+schema_version = 1
+implementation_status = "planned"
+
+[targets]
+systems = ["x86_64-linux", "aarch64-linux"]
+artifacts = ["linux-x86_64", "linux-aarch64"]
+format = "relocatable-archive"
+
+[support]
+primary = ["native-linux-x86_64", "native-linux-aarch64"]
+secondary = ["wsl2-x86_64", "wsl2-aarch64"]
+unsupported = ["wsl1", "native-windows"]
+validation_status = "planned-not-yet-verified"
+wsl_uses_linux_artifacts = true
+wsl_installation_is_user_prerequisite = true
+wsl_recommended_storage = "linux-filesystem"
+
+[offline]
+first_launch_downloads = false
+requires_host_guix = false
+requires_root = false
+include_application = true
+include_language_runtimes = true
+include_scientific_tools = true
+include_julia_packages_and_artifacts = true
+include_r_packages = true
+include_web_assets = true
+include_reference_databases = false
+include_user_sequencing_data = false
+
+[tooling]
+environment_builder = "guix"
+version_management = "mise"
+task_runner = "just"
+javascript_toolchain = "bun"
+bundled = ["mise", "just", "bun"]
+prohibited_toolchain_commands = ["npm", "pnpm", "deno"]
+new_application_typescript = false
+
+[updates]
+mode = "automatic-when-idle"
+unit = "complete-tested-release"
+version_target = "latest-compatible-stable"
+component_updates_in_place = false
+requires_signed_metadata = true
+requires_artifact_hashes = true
+requires_rollback = true
+requires_idle_gate = true
+requires_health_check = true
+startup_waits_for_network = false
+update_failure_blocks_existing_release = false
+allow_prereleases = false
+
+[acceptance]
+network_disabled_first_launch = true
+clean_host_without_application_tools = true
+both_architectures_required = true
+preserve_existing_features = true
+incremental_migration_allowed = true