watchdog in daslang: a process handle in fio, whole-program standalone emission, watchdog.py retired

.github/workflows/extended_checks.yml

@@ -564,13 +564,6 @@ jobs:
         python3 ci/test_fix_md_ascii.py
         python3 ci/fix_md_ascii.py --check
 
-    - name: "Test watchdog tray wording (python)"
-      if: matrix.role != 'modules'
-      run: |
-        set -eux
-        python3 utils/watchdog/test_tray_state.py
-        python3 utils/watchdog/test_consent.py
-
     - name: "Test pip wheel repack (python)"
       if: matrix.role != 'modules'
       run: |

CHANGELIST.md

@@ -232,6 +232,9 @@ Z3 SMT solver bindings as a dynamic module, dasLLVM-style.
 - **JIT debug tooling** (#3511); **`llvm_tune` per-box UX** (#3403) - scope / policy / `--tune`, self-tuning servers, `-exe` fix
 - **AOT batch-composition hash fix** (`g_isInAot` leak) + `aot_cpp` made AOT-linkable (#3409); **AOT fuzzer-failure hardening** (#3303)
 - **Standalone contexts link C++ modules** (#3947) - a `-ctx` context that reaches dasHV, fio or any handled type registers the modules it links (the builtin set in the C++ registrar's order, then dependencies-first) on first construction through a process-wide list (`include/daScript/simulate/standalone_modules.h`), so several generated contexts in one binary share one registry lifetime that ends with `Module::ShutdownStandalone` when the last context is destroyed; a builtin module the program calls at run time keeps its AOT header, the function table is dense (a slot per function the context still reaches), and `examples/standalone/06_full_runtime` (dasHV + fio, static, compiles nothing at run time, loads no shared module) is the worked example and a small-lane test
+- **Standalone contexts emit every used function of every module** (#3950) - a `-ctx` context is one translation unit, so a required das module's functions, its class methods (through their Func slots) and the modules their externs reach now land in it; a generic instance several modules instantiated is emitted once per name. The entry module alone was emitted before, which the example survived only because its one cross-module das call was inlined.
+- **`fio`: a long-lived child process** (#3950) - `spawn_process` / `process_drain` / `process_poll` / `process_wait` / `process_terminate` / `process_kill` / `process_pid` / `process_alive` / `close_process`, the `process_running` sentinel, and `with_process` in `daslib/fio`: an opaque `SubProcess?` a supervisor drives on its own clock, with a non-blocking line drain over merged stdout+stderr, `KEY=VALUE` environment overrides, a working directory, and tree-wide signals (a Windows job object, a POSIX process group). `popen_argv` stays the block-scoped, run-to-completion form.
+- **The watchdog is daslang, and a static executable** (#3950) - `utils/watchdog/watchdog.py` (and its Python dependency in every release bundle) is replaced by `utils/watchdog/watchdog.das` plus `bin/watchdog`, a standalone context on the full runtime with dasHV static: no compiler, no shared module, no lock on the files a deploy replaces. Config, `watchdog.json` layering, layout discovery, the exit 0 / 3 / 4 contract, startup stages, the JSON-lines log with rotation, crash bundles, WER and notifications carry over; the exchange consent dialog, the sidecar tray rails and the tray icon do not. Manifests ship it with the new `release_include_tool("watchdog")`, which resolves `bin/watchdog` or `bin/Release/watchdog.exe` per platform.
 - **LLVM-AOT in a large embedding host** (#3715) - scalar call ABI matched at bool/reference seams, target triple + data layout pinned on emitted objects, `-dll-path`/`DAS_DLL_PATH` dasbind search, per-object glob-init deferred to first link
 
 #### Runtime, Tooling, and Hosting

CLAUDE.md

@@ -238,6 +238,9 @@ diagnostic in any tier.
   fresh clone per registration, not the same variable passed twice.
 - **`new WithCtor(field = v)` skips the user constructor** - it is plain field-init, so
   inherited fields stay zero. Write `new WithCtor(args)` when the constructor must run.
+- **`exit(N)` does not set the process exit code under the daslang CLI.** It unwinds as an
+  abnormal termination and the process reports 1, whatever `N` was - a supervisor or shell sees a
+  crash. A code the parent must read comes from `def main() : int { return N }`.
 
 ### Code style - prefer idiomatic forms
 

CMakeLists.txt

@@ -2136,11 +2136,11 @@ install(FILES ${PROJECT_SOURCE_DIR}/utils/dasllama-convert/main.das
     DESTINATION utils/dasllama-convert
 )
 
-# Install watchdog (supervisor for long-running daslang programs; the worked
-# example in the daspkg skill's release_include_from docs)
+# Install watchdog (supervisor for long-running daslang programs; the static exe installs
+# beside daslang from utils/CMakeLists.txt, these are the library and the interpreter entry)
 install(FILES
-    ${PROJECT_SOURCE_DIR}/utils/watchdog/watchdog.py
-    ${PROJECT_SOURCE_DIR}/utils/watchdog/dummy_server.py
+    ${PROJECT_SOURCE_DIR}/utils/watchdog/watchdog.das
+    ${PROJECT_SOURCE_DIR}/utils/watchdog/main.das
     ${PROJECT_SOURCE_DIR}/utils/watchdog/README.md
     DESTINATION utils/watchdog
 )

ci/test_check_shipped_skills.py

@@ -109,7 +109,7 @@ def test_utils_daslang_path_fires_watchdog_passes(self):
               "# Good\n\nsee utils/daslang/main.cpp for the gc hook\n")
         self.assert_fires("not in bundle", "utils/daslang/main.cpp")
         write(self.bundle, "skills/good.md",
-              "# Good\n\nrun `python utils/watchdog/watchdog.py`\n")
+              "# Good\n\nrun `daslang utils/watchdog/main.das -- --cwd .`\n")
         rc, out = self.run_gate()
         self.assertEqual(rc, 0, out)
 

daslib/ARCHITECTURE_EMIT.md

@@ -83,10 +83,20 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across
   index. `[init]` function order is not re-derived at all: the emitter reads the
   simulated context's list through rtti `for_each_init_function`, so the C++ late-init
   sort stays the single source of truth.
-- **Cross-module limits fail loud at emit time**: only main-module, AOT-emitted `[init]`
-  functions can be called from the ctor (required-module and `[no_aot]` ones are collected
-  emit errors with the reason), because the standalone TU only emits the entry module's
-  function bodies.
+- **The TU holds every used function of every module.** A standalone context is one
+  translation unit with no other to link, so `prepareProgramForEmission`'s markers, the
+  block-variable collector, `registerAotCpp`'s `ArgsConverter` thunks and
+  `StandaloneContextGen` each walk the entry module through `visit_module` and then every
+  foreign used function through `visit(fn, adapter)` (`foreignUsedFunctions`, aot_cpp): a
+  builtin is C++ already, a template never emits, a used `[no_aot]` function was refused by
+  name before the walk. A generic instance several modules instantiated has one AOT name,
+  so declarations, bodies, thunks and table rows are emitted once per name. A foreign
+  global's initializer temporaries collect under the collector's null key, the one
+  `__init_script` declares. `UseTypeMarker` walks the foreign functions too, so an extern
+  or handled type reached only from one still links its module.
+- **Cross-module `[init]` is refused at emit time**: only entry-module, AOT-emitted `[init]`
+  functions are called from the ctor; a required module's and `[no_aot]` ones are collected
+  emit errors with the reason.
 - **Every used function must have an AOT body** - a standalone context has no
   interpreter, so a used `noAot` function (the `[no_aot]` annotation, or `NoAotMarker`
   finding a type AOT cannot express) is a collected emit error, never a

daslib/REVIEW.md

@@ -137,10 +137,11 @@ spell it are `aotStructName` and the `VarInfo` emitter's inline
 `aotSuffixNameEx(info.name, "_S", ...)`. One site changed alone writes `offsetof`s that
 name a struct declared under a different name.
 
-**A diff that adds or changes a function that runs `CppAot` or any subclass of it
-keeps `buildStructEnumCollisions` running before that visitor runs - directly or in a
-helper it calls.** The table decides when a name gets its collision suffix, and a run
-that skips the seeding spells structs differently from the run that seeded it.
+**A diff that adds or changes a function that emits a struct or enum C++ name - anything
+reaching `aotStructName` / `aotEnumName`, a `CppAot` subclass and `ArgsConverter` alike -
+keeps `buildStructEnumCollisions` running before it, directly or in a helper it calls.**
+The table decides when a name gets its collision suffix, and a run that skips the seeding
+spells structs differently from the run that seeded it.
 
 **Never pass a synthesized access expression's location to `match_error` - pass a pattern
 node's location.** `match_error` stores the `LineInfo` pointer BORROWED, and access nodes

daslib/aot_cpp.das

@@ -4214,6 +4214,12 @@ def public dumpDependencies(program : ProgramPtr; var aotVisitor : CppAot?; prun
     var utm = new UseTypeMarker();
     make_visitor(*utm) $(adapter) {
         visit(program, adapter);
+        if (prune_to_used) {                       // a standalone context emits the foreign functions too
+            var foreign <- foreignUsedFunctions(program, false)
+            for (fn in foreign) {
+                visit(fn, adapter)
+            }
+        }
     }
     let remUS = program._options  |> find_arg("remove_unused_symbols") ?as tBool ?? true;
     let kept <- prune_to_used ? collectUsedModules(program) : table<Module?>()
@@ -4337,6 +4343,23 @@ def collectUsedFunctions(modules : array<Module?>; totalFunctions : int; this_mo
     return <- fnn;
 }
 
+//! The used functions living outside the entry module. A standalone context is one translation
+//! unit, so each is emitted there or nowhere - builtins are C++ already, templates never emit.
+//! `before_no_aot_marking` keeps the ones NoAotMarker has yet to judge; after it, `[no_aot]` is out.
+def public foreignUsedFunctions(program : ProgramPtr; before_no_aot_marking : bool) : array<Function?> {
+    let thisModule = program.getThisModule
+    var foreign : array<Function?>
+    program.get_ptr() |> for_each_module_no_order($(pm) {
+        if (pm == thisModule) return
+        pm |> for_each_module_function($(pfun) {
+            if (pfun.index < 0 || !pfun.flags.used || pfun.flags.builtIn || pfun.moreFlags.isTemplate
+                    || (!before_no_aot_marking && pfun.flags.noAot)) return
+            foreign |> push(pfun)
+        })
+    })
+    return <- foreign
+}
+
 def public collectProgramUsedFunctions(program : ProgramPtr; all_modules : bool; is_all : bool) : array<Function?> {
     //! Collects all used functions from a program's modules for AOT code generation.
     var modules : array<Module?>;
@@ -4351,6 +4374,21 @@ def registerAotCpp(var logs : StringBuilderWriter?; program : ProgramPtr; var co
     var visitor = new ArgsConverter(logs, cross_platform);
     make_visitor(*visitor) $(adapter_marker) {
         visit(program, adapter_marker);
+        if (all_modules) {
+            // a whole-program table (a standalone context) needs a thunk per foreign row as well;
+            // a generic instance several modules share has one name and gets one thunk
+            var thunked : table<string>
+            for (pfun in collectProgramUsedFunctions(program, false, false)) {
+                thunked |> insert(aotFuncName(pfun))
+            }
+            var foreign <- foreignUsedFunctions(program, false)
+            for (fn in foreign) {
+                let name = aotFuncName(fn)
+                if (thunked |> key_exists(name)) continue
+                thunked |> insert(name)
+                visit(fn, adapter_marker)
+            }
+        }
     }
     unsafe {
         delete visitor
@@ -4360,9 +4398,12 @@ def registerAotCpp(var logs : StringBuilderWriter?; program : ProgramPtr; var co
         write(*logs, "\n#ifdef _MSC_VER\n#pragma optimize(\"\", off)\n#endif\n");
         write(*logs, "struct AotFunction \{ uint64_t hash; bool is_cmres; void * fn; vec4f (*wrappedFn)(Context*); \};\n");
         write(*logs, "static AotFunction functions[] = \{\n");
+        var listed : table<string>
         for (fn in fnn) {
             let is_cmres = fn.flags.copyOnReturn || fn.flags.moveOnReturn ? "true" : "false";
             let fn_name = aotFuncName(fn);
+            if (listed |> key_exists(fn_name)) continue
+            listed |> insert(fn_name)
             write(*logs, "    // {get_aot_hash_comment(fn)}\n");
             write(*logs, "    \{ {fn.aotHash}, {is_cmres}, (void*)&{fn_name}, &__wrap_{fn_name} \},\n");
         }
@@ -4448,6 +4489,10 @@ def private collectUsedModules(program : ProgramPtr) : table<Module?> {
     var utm = new UseTypeMarker()
     make_visitor(*utm) $(adapter) {
         visit(program, adapter)
+        var foreign <- foreignUsedFunctions(program, false)
+        for (fn in foreign) {
+            visit(fn, adapter)
+        }
     }
     for (st in keys(utm.useStructs)) {
         if (st._module != null) {

daslib/aot_standalone.das

@@ -129,7 +129,7 @@ def writeStandaloneCtor(cfg : StandaloneContextCfg; initFunctions : string; var
     let requested_stack = stack_arg ?as tInt ?? int(program.policies.stack)
     let stack_base = requested_stack > 0 && stack_arg is tInt ? requested_stack : max(requested_stack, min_init_stack)
     let stack_size = stack_base + int(program.globalInitStackSize)
-    let usedFunctionCount = length(collectProgramUsedFunctions(program, false, false))
+    let usedFunctionCount = length(collectProgramUsedFunctions(program, true, false))
     write(tw, "{cfg.class_name}::{cfg.class_name}() : Context({stack_size}/*stack*/) \{\n");
     write(tw, "    auto & context = *this;\n");
     write(tw, "    CodeOfPolicies policies;");
@@ -257,14 +257,15 @@ class StandaloneContextGen : CppAot {
         declarations = ss |> string_builder_str();
         ss |> string_builder_clear();
         write(*ss, "\n");
-        let fnn = collectProgramUsedFunctions(prog, false, false);
+        // every used function of every module, declared once (a shared generic instance has one name)
+        let fnn = collectProgramUsedFunctions(prog, true, false);
         var inline_fns : array<string>
+        inline_fns |> reserve(length(fnn))
         for (pfun in fnn) {
-            let needInline = that == pfun._module;
-            if (needInline) {
-                inline_fns.push(describeCppFunc(pfun, collector, cross_platform, true, needInline));
-                used_functions.insert(aotFuncName(pfun));
-            }
+            let name = aotFuncName(pfun)
+            if (used_functions |> key_exists(name)) continue
+            inline_fns.push(describeCppFunc(pfun, collector, cross_platform, true, true));
+            used_functions.insert(name);
         }
         let sep = ";\n";
         let maybe_sem = inline_fns |> empty() ? "" : sep;
@@ -308,7 +309,7 @@ def writeRegistration(var header : StringBuilderWriter;
     write(source, "using namespace {program.thisNamespace};\n");
     write(header, "namespace {cfg.context_name} \{\n");
     write(source, "namespace {cfg.context_name} \{\n");
-    dumpRegisterAot(unsafe(addr(source)), program, context, false, cfg.cross_platform);
+    dumpRegisterAot(unsafe(addr(source)), program, context, true, cfg.cross_platform);
     writeModuleRegistration(source, registrations);
     writeStandaloneContext(program, initFunctions, header, source, cfg, context);
     write(header, "\} // namespace {cfg.context_name}\n");
@@ -374,9 +375,21 @@ def genStandaloneSrc(var program : ProgramPtr;
             make_visitor(*gen) $(adapter) {
                 gen.adapter := adapter
                 program |> visit_module(adapter, program.getThisModule);
+                // a generic instance several modules instantiated is one C++ function: emit the first copy only
+                var emitted : table<string>
+                for (pfun in collectProgramUsedFunctions(program, false, false)) {
+                    emitted |> insert(aotFuncName(pfun))
+                }
+                var foreign <- foreignUsedFunctions(program, false)
+                for (fn in foreign) {
+                    let name = aotFuncName(fn)
+                    if (emitted |> key_exists(name)) continue
+                    emitted |> insert(name)
+                    visit(fn, adapter)
+                }
             }
 
-            initFunctions = addFunctionInfo(collectProgramUsedFunctions(program, false, false), gen.helper);
+            initFunctions = addFunctionInfo(collectProgramUsedFunctions(program, true, false), gen.helper);
             type_defs += gen.declarations;
             gen.declarations = "";
             write(tw, gen.str());
@@ -421,21 +434,32 @@ def private checkAllUsedFunctionsCanAot(program : ProgramPtr) {
 
 def private prepareProgramForEmission(var program : ProgramPtr; context : Context) : BlockVariableCollector? {
     var noAotMarker = new NoAotMarker();
+    var unmarked <- foreignUsedFunctions(program, true)
     make_visitor(*noAotMarker) $(adapter_no_aot) {
         visit(program, adapter_no_aot)
+        for (fn in unmarked) {
+            visit(fn, adapter_no_aot)
+        }
     }
     checkAllUsedFunctionsCanAot(program)
+    var foreign <- foreignUsedFunctions(program, false)
 
     var pmarker = new PrologueMarker();
     make_visitor(*pmarker) $(adapter_p) {
         visit(program, adapter_p)
+        for (fn in foreign) {
+            visit(fn, adapter_p)
+        }
     }
 
     setAotHashes(program, context);
 
     var flags = new SetPrinterFlags();
     make_visitor(*flags) $(flags_adapter) {
         visit(program, flags_adapter)
+        for (fn in foreign) {
+            visit(fn, flags_adapter)
+        }
     }
     unsafe {
         delete flags
@@ -446,6 +470,18 @@ def private prepareProgramForEmission(var program : ProgramPtr; context : Contex
     var coll = new BlockVariableCollector();
     make_visitor(*coll) $(adapter_coll) {
         visit(program, adapter_coll)
+        for (fn in foreign) {
+            visit(fn, adapter_coll)
+        }
+        // foreign globals initialize in __init_script too: their temporaries collect under the null key
+        let thisModule = program.getThisModule
+        program.get_ptr() |> for_each_module_no_order($(pm) {
+            if (pm == thisModule) return
+            pm |> for_each_global($(pvar) {
+                if (pvar.index < 0 || !pvar.flags.used || pvar.init == null) return
+                visit_expression(pvar.init, adapter_coll)
+            })
+        })
     }
     return coll
 }