Add tests (bring #17 and #18 to main)

.github/workflows/test.yml

@@ -0,0 +1,56 @@
+name: Tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+    inputs:
+      node-red-ref:
+        description: "node-red branch, tag or sha to compare node-red-types and the editor fixture with"
+        required: false
+        default: "master"
+
+permissions:
+  contents: read
+
+jobs:
+  unit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npm test
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npm run test:build
+      - run: npx playwright install --with-deps chromium
+      - run: npm run test:browser
+
+  node-red-sync:
+    # Depends on node-red's state, not just this repo, so it only runs on request.
+    if: github.event_name == 'workflow_dispatch'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npm run test:node-red-sync
+        env:
+          NODE_RED_REF: ${{ inputs.node-red-ref }}

README.md

@@ -14,6 +14,7 @@ This project makes an ESM bundle of monaco-editor with 50 themes and localizatio
 
 ## Notes
 * A bug / issue I had to handle is: when changing mode, html worker would attempt to get links from document, but occasionally `document` would be `null` and an exception would be thrown. This has been handled by adding `if(!document) return []` at the top of `function findDocumentLinks(...)` in file `htmlLinks.js`
+* The build adds `monaco.css.getWorker(languageId)` and `monaco.html.getWorker(languageId)`, like monaco's own `monaco.json.getWorker()`, so the editor can use the CSS and HTML language services for embedded content (e.g. a `<style>` block in HTML). `languageId` defaults to `css` / `html`; `scss`, `less`, `handlebars`, `razor` and ids passed to `monaco.html.registerHTMLLanguageService()` work too. Each resolves to `(...uris) => Promise<worker>` once that language has been used. See `exposeLanguageWorkers()` in `generate-monaco-esm-i18n.js`; the build fails if monaco's code has moved and the patch no longer applies. A language added with `registerHTMLLanguageService()` asks `MonacoEnvironment.getWorkerUrl` for a worker labelled with its own id, so map that label to `html` there.
 
 ## Instructions
 
@@ -39,6 +40,8 @@ Check + update node-red (function node/server-side) type defs
 * `./node-red-types/func.d.ts`
 * `./node-red-types/util.d.ts`
 
+Make type changes here first, or alongside the node-red PR that needs them. `npm run copy` puts them into node-red. If someone changed node-red's copies directly (`editor-client/src/types/node-red/`), bring that change back here before building, or the next copy undoes it. `npm run test:node-red-sync` shows any difference (see [Compare with node-red](#compare-with-node-red)).
+
 ### Step 3
 
 Install dependencies, clean and build
@@ -55,7 +58,37 @@ This will bundle the monaco editor with localization support and themes:
 cd output/monaco/dist/
 ```
 
-### Check it works
+### Check the build before copying it to node-red
+
+The output is vendored into node-red as is, so check it first. A build can succeed and still give node-red types that resolve to `any`, a worker that never loads or a locale that translates nothing, with nothing in the build log.
+
+```bash
+npm test               # the types, as node-red's Function node editor loads them (no build needed)
+npm run test:build     # clean + build, then everything again against output/, plus bundle, locale and theme checks
+npm run test:browser   # the built editor in headless Chromium (run `npx playwright install chromium` once)
+```
+
+`npm test` loads the types the way node-red's `monaco.js` does: one `.d.ts` per module, from the list in `test/fixtures/editor-modules.js`. It checks:
+
+* default globals like `Buffer.from()`, `NodeJS.TypedArray`, `node.send()` and `RED.util` resolve to real types, not `any`
+* every module in node-red's `knownModules` resolves when added on the Setup tab
+* the exclude list in `update-types.js` still matches real files, and nothing it excludes ships
+* types that already fail to resolve, and the list of shipped files, match `test/baselines/`
+
+If a bump changes which files node-red has to load (for example @types/node 24 moved `Buffer.from()` into `buffer.buffer.d.ts`), the tests fail on whatever turned into `any`. Update `knownModules`/`defaultServerSideTypes` in node-red's `monaco.js` in a node-red PR alongside this one, and make the same change in `test/fixtures/editor-modules.js`.
+
+When a bump changes a baseline on purpose, run `npm run test:update-baseline` and review the diff before committing it.
+
+#### Compare with node-red
+
+```bash
+NODE_RED_DIR=../node-red npm run test:node-red-sync   # a local checkout, on whatever branch it is on
+NODE_RED_REF=dev npm run test:node-red-sync           # or a branch/tag on GitHub (default master)
+```
+
+Compares `node-red-types/` with node-red's copies, to catch changes made directly in node-red that need bringing back here. Also compares `test/fixtures/editor-modules.js` with the `monaco.js` this build is going into. Differences are expected while the matching node-red PR is still open.
+
+#### Check it by eye
 
 Check editor works in browser...
 

demo.html

@@ -168,7 +168,9 @@ <h3>Things to check...</h3>
                 types + "node-red/util.d.ts",
                 types + "node-red/func.d.ts",
                 types + "node/globals.d.ts",
+                types + "node/globals.typedarray.d.ts",
                 types + "node/buffer.d.ts",
+                types + "node/buffer.buffer.d.ts",
                 types + "node/console.d.ts",
                 types + "node/fs.d.ts",
                 types + "node/fs/promises.d.ts",

generate-monaco-esm-i18n.js

@@ -120,6 +120,12 @@ function injectSourcePath(monacoVersion, callback) {
                         }
                     }
                 });
+                try {
+                    exposeLanguageWorkers();
+                } catch (err) {
+                    callback(err);
+                    return;
+                }
                 callback();
             });
         });
@@ -128,6 +134,77 @@ function injectSourcePath(monacoVersion, callback) {
     })
 }
 
+/**
+ * Replace text in a file and fail the build unless it matched exactly `expected` times, so a monaco
+ * update that moves the code can't silently drop the patch.
+ */
+function patchExactly(file, from, to, expected = 1) {
+    // replace-in-file counts with String.match(), which only counts every match for a global regex
+    from = from.global ? from : new RegExp(from.source, from.flags + "g");
+    const [result] = replaceInFileSync({ files: file, from, to, countMatches: true });
+    const found = result ? result.numMatches : 0;
+    if (found !== expected) {
+        throw new Error(`nr-monaco-build: expected ${expected} match(es) of ${from} in ${file} but found ${found}. ` +
+            "monaco has changed, update exposeLanguageWorkers() in generate-monaco-esm-i18n.js");
+    }
+}
+
+/**
+ * Expose the CSS and HTML language workers, the same way monaco already does for JSON
+ * (`monaco.json.getWorker()`) and TypeScript (`monaco.typescript.getJavaScriptWorker()`), so editor
+ * code can use those language services for embedded content, e.g. a <style> block inside HTML:
+ *
+ *     monaco.css.getWorker(languageId = "css")    // "css", "scss" or "less"
+ *     monaco.html.getWorker(languageId = "html")  // "html", "handlebars", "razor" or an id passed to
+ *                                                  // monaco.html.registerHTMLLanguageService()
+ *
+ * Each resolves to `(...uris) => Promise<worker>`, like `monaco.json.getWorker()`, and rejects with
+ * "<languageId> not registered!" until that language has been used (its worker starts lazily).
+ * The worker methods (doComplete, doHover, doValidation, findDocumentColors, ...) are monaco's
+ * CSSWorker / HTMLWorker classes in esm/vs/languages/features/{css,html}/*Worker.js.
+ */
+function exposeLanguageWorkers() {
+    const registry = [
+        "",
+        "// Added by nr-monaco-build: expose the language workers, like monaco.json.getWorker()",
+        "const nrWorkers = new Map();",
+        "function nrRegisterWorker(languageId, worker, disposables) {",
+        "  nrWorkers.set(languageId, worker);",
+        "  disposables?.push({ dispose: () => { if (nrWorkers.get(languageId) === worker) { nrWorkers.delete(languageId); } } });",
+        "}",
+        "function getWorker(languageId) {",
+        "  return new Promise((resolve, reject) => {",
+        "    const worker = nrWorkers.get(languageId);",
+        "    if (!worker) {",
+        "      return reject(languageId + \" not registered!\");",
+        "    }",
+        "    resolve(worker);",
+        "  });",
+        "}",
+        ""
+    ].join("\n");
+    const featuresDir = path.join(monacoModEsmDir, "vs", "languages", "features");
+    // htmlMode.js has two setup functions (setupMode and the older setupMode1), cssMode.js has one
+    for (const [lang, setupCount] of [["css", 1], ["html", 2]]) {
+        const modeFile = path.join(featuresDir, lang, `${lang}Mode.js`);
+        // 1. remember each language's worker accessor when its mode is set up
+        patchExactly(modeFile,
+            /(\n[ \t]*const worker = \(\.\.\.uris\) => \{\s*return client\.getLanguageServiceWorker\(\.\.\.uris\);\s*\};)/g,
+            `$1\n  nrRegisterWorker(defaults.languageId, worker, typeof disposables === "undefined" ? void 0 : disposables);`,
+            setupCount);
+        // 2. add the registry and export getWorker
+        patchExactly(modeFile, /\nexport \{ ([^}]*) \};\s*$/, `\n${registry}\nexport { $1, getWorker };\n`);
+        // 3. public API: monaco.css.getWorker / monaco.html.getWorker
+        patchExactly(path.join(featuresDir, lang, "register.js"), /\nexport \{ ([^}]*) \};\s*$/,
+            `\nconst getWorker = (languageId = "${lang}") => getMode().then((mode2) => mode2.getWorker(languageId));\nexport { $1, getWorker };\n`);
+        // 4. older import path (esm/vs/language/<lang>/monaco.contribution.js) re-exports by name
+        const contribution = path.join(monacoModEsmDir, "vs", "language", lang, "monaco.contribution.js");
+        if (fs.existsSync(contribution)) {
+            patchExactly(contribution, /export \{ ([^}]*) \} from '([^']*\/register\.js)';/, "export { $1, getWorker } from '$2';");
+        }
+    }
+}
+
 
 async function main() {
     const pkg = await readPackage("package.json");

package.json

@@ -18,6 +18,11 @@
         "pack-dev": "npx webpack --mode development",
         "demo": "npx http-server -p 8080 -c-1 -o demo.html",
         "generate-extraLibs": "node generate-extraLibs.js",
+        "test": "node test/run.js",
+        "test:update-baseline": "node test/run.js --update-baseline",
+        "test:build": "npm run clean && npm run build && node test/run.js --output",
+        "test:node-red-sync": "node test/run.js --node-red-sync",
+        "test:browser": "node test/run.js --browser",
         "copyCheck1": "test -d output/monaco/dist",
         "copyCheck2": "test -d ../node-red/packages/node_modules/@node-red/editor-client/src",
         "copyPrep1": "rm -rf ../node-red/packages/node_modules/@node-red/editor-client/src/vendor/monaco/dist/*",
@@ -44,6 +49,7 @@
         "monaco-editor-webpack-plugin": "^7.1.1",
         "monaco-themes": "^0.4.7",
         "ncp": "^2.0.0",
+        "playwright": "^1.64.0",
         "read-package-json": "^7.0.1",
         "recursive-readdir": "^2.2.3",
         "replace-in-file": "^6.3.2",

test/baselines/shipped-files.json

@@ -0,0 +1,72 @@
+{
+    "node": [
+        "assert.d.ts",
+        "assert/strict.d.ts",
+        "async_hooks.d.ts",
+        "buffer.buffer.d.ts",
+        "buffer.d.ts",
+        "child_process.d.ts",
+        "cluster.d.ts",
+        "compatibility/iterators.d.ts",
+        "console.d.ts",
+        "crypto.d.ts",
+        "dgram.d.ts",
+        "diagnostics_channel.d.ts",
+        "dns.d.ts",
+        "dns/promises.d.ts",
+        "domain.d.ts",
+        "events.d.ts",
+        "fs.d.ts",
+        "fs/promises.d.ts",
+        "globals.d.ts",
+        "globals.typedarray.d.ts",
+        "http.d.ts",
+        "http2.d.ts",
+        "https.d.ts",
+        "module.d.ts",
+        "net.d.ts",
+        "os.d.ts",
+        "path.d.ts",
+        "perf_hooks.d.ts",
+        "process.d.ts",
+        "querystring.d.ts",
+        "readline.d.ts",
+        "readline/promises.d.ts",
+        "sea.d.ts",
+        "sqlite.d.ts",
+        "stream.d.ts",
+        "stream/consumers.d.ts",
+        "stream/promises.d.ts",
+        "stream/web.d.ts",
+        "string_decoder.d.ts",
+        "test.d.ts",
+        "timers.d.ts",
+        "timers/promises.d.ts",
+        "tls.d.ts",
+        "trace_events.d.ts",
+        "ts5.6/buffer.buffer.d.ts",
+        "ts5.6/compatibility/float16array.d.ts",
+        "ts5.6/globals.typedarray.d.ts",
+        "ts5.7/compatibility/float16array.d.ts",
+        "tty.d.ts",
+        "url.d.ts",
+        "util.d.ts",
+        "v8.d.ts",
+        "vm.d.ts",
+        "wasi.d.ts",
+        "web-globals/abortcontroller.d.ts",
+        "web-globals/crypto.d.ts",
+        "web-globals/domexception.d.ts",
+        "web-globals/events.d.ts",
+        "web-globals/fetch.d.ts",
+        "web-globals/navigator.d.ts",
+        "web-globals/storage.d.ts",
+        "web-globals/streams.d.ts",
+        "worker_threads.d.ts",
+        "zlib.d.ts"
+    ],
+    "node-red": [
+        "func.d.ts",
+        "util.d.ts"
+    ]
+}