Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
push:
pull_request:

permissions:
contents: read

jobs:
test:
strategy:
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,17 @@ name: Security

on:
push:
branches: [master]
branches: [main]
pull_request:
branches: [master]
branches: [main]
schedule:
# Weekly, to catch newly disclosed CVEs even when nothing in the repo
# has changed. CodeQL has its own schedule in codeql.yml.
- cron: '0 6 * * 1'

permissions:
contents: read

jobs:
npm-audit:
name: npm audit
Expand Down
71 changes: 71 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

This fork ([`cld-universal`](https://github.com/figulusproject/cld-universal)) diverged from
[`dachev/node-cld`](https://github.com/dachev/node-cld) at `2.10.1`. Entries below start from that
point.

## [2.11.1] - 2026-08-03

### Added

- `setWasmModuleOptions({ locateFile })` in the browser entry point, letting bundlers/apps override
where the WASM backend fetches `cld.web.wasm` from when it isn't served at the default relative
path.

### Fixed

- Browser entry point's `metadata.json` import now uses the `with { type: 'json' }` attribute
required by Node's ESM loader.

### Security

- Added a CodeQL Advanced workflow (`codeql.yml`) analyzing the `actions`, `c-cpp`, and
`javascript-typescript` languages on push/PR to `main` and weekly on a schedule.
- Added a `security.yml` workflow running `npm audit --audit-level=high` against the package's
npm dependencies on push/PR to `main` and weekly on a schedule, separate from the CodeQL
workflow above.
- Restricted the default `GITHUB_TOKEN` permissions to `contents: read` on the `ci.yml` and
`security.yml` workflows.

## [2.11.0] - 2026-08-02

### Added

- WASM fallback backend: the native addon is tried first, and the package now falls back to a
bundled WASM build (built via Emscripten from the same CLD2 sources) when the native addon is
unavailable, including in the browser and in environments without a C++ toolchain.
- GitHub Actions CI covering both the native and WASM backends across Node 20/22, replacing the
old Travis/AppVeyor setup.
- npm publish workflow.

### Changed

- Package renamed from `cld` to `cld-universal`; metadata, description, keywords, and
repository/bugs/homepage links updated to point at this fork.
- README rewritten to document the fork's scope, maintenance expectations, and the WASM fallback.
- Dropped support for Node 8/10/12; `engines` now requires `20 || >=22`.
- `postinstall.js` updated to use `globSync` (required by the `glob` upgrade below).
- Install step (`bin/build-native.js`) now always exits `0` after attempting `node-gyp rebuild`, so
a missing or broken C++ toolchain no longer fails `npm install` before the WASM fallback is
reachable.
- `package.json` now declares an explicit `files` field (previously unset, so publishing silently
fell back to `.gitignore` rules).

### Fixed

- Missing emsdk setup step in the publish workflow.
- AppVeyor/Windows (MSVC) build failures: pinned `node-addon-api` to `^2.0.0`, moved
`NODE_API_MODULE` out of the `NodeCld` namespace, and bumped the CI image to Visual Studio 2022.

### Security

- Updated `glob` from `7.x` to `^12` (fixes 2 high severity advisories, including the
brace-expansion DoS in [GHSA-mh99-v99m-4gvg](https://github.com/advisories/GHSA-mh99-v99m-4gvg))
while retaining support for Node < 20.
- Updated `underscore` from `^1.12.1` to `^1.13.7` (fixes a high severity advisory).
- Updated `minimatch` from `3.0.4` to `3.1.2`.
- Migrated the lockfile from v1 to v3.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,17 @@ const result = await detect('This is a language recognition example');
console.log(result);
```

If your bundler serves the WASM binary from a non-default path (a CDN, a renamed/hashed asset, etc.), call `setWasmModuleOptions({ locateFile })` before the first `detect()` call to override where `cld.web.wasm` is fetched from:
```js
import { detect, setWasmModuleOptions } from 'cld-universal';

setWasmModuleOptions({
locateFile: (path) => `/assets/${path}`,
});

const result = await detect('This is a language recognition example');
```

## Result shape

`detect()` resolves to (or, in callback form, passes as the second argument) an object of the following shape:
Expand Down
5 changes: 5 additions & 0 deletions index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ export const LANGUAGES: {[name: string]: string};
export const DETECTED_LANGUAGES: string[];
export const ENCODINGS: string[];

// Browser entry point only (./wasm/browser-entry.js). Overrides where the
// WASM backend fetches cld.web.wasm from -- has no effect on the native
// backend. Must be called before the first detect() call.
export declare function setWasmModuleOptions(options: { locateFile?: (path: string, scriptDirectory: string) => string }): void;

export declare function detect(text: string, options: Options, callback: (err: string, result: DetectLanguage) => void): void;
export declare function detect(text: string, callback: (err: string, result: DetectLanguage) => void): void;
export declare function detect(text: string, options: Options): Promise<DetectLanguage>;
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"wasm",
"webassembly"
],
"version": "2.11.0",
"version": "2.11.1",
"main": "./index.js",
"types": "./index.d.ts",
"exports": {
Expand Down Expand Up @@ -43,7 +43,7 @@
"install": "node bin/build-native.js",
"build": "node-gyp rebuild",
"build:wasm": "scripts/build-wasm.sh",
"test": "node test/runner.js && node test/runner-wasm.js",
"test": "node test/runner.js && node test/runner-wasm.js && node test/runner-wasm-browser.js",
"postinstall": "node bin/postinstall.js"
},
"author": {
Expand Down
76 changes: 76 additions & 0 deletions test/runner-wasm-browser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/usr/bin/env node
// Proves wasm/browser-entry.js's setWasmModuleOptions({ locateFile }) hook
// actually controls where the WASM backend fetches cld.web.wasm from --
// not just that the option is accepted, but that redirecting it to a
// custom URL is what makes detection succeed.
//
// Each case runs in its own child process: browser-entry.js caches
// moduleOptions/modulePromise at module scope on first use, so "no
// override" and "with override" can't share one process/import.
//
// With no override, browser-entry.js's default WASM URL resolves against
// import.meta.url, which is a file:// URL here -- and Node's fetch()
// doesn't support file://, so detect() is expected to reject. That's not a
// Node quirk being worked around, it's exactly the gap the override exists
// for, and it doubles as proof that the override (not some other
// already-working path) is what makes the second case below succeed.

const assert = require('assert');
const http = require('http');
const path = require('path');
const fs = require('fs');
const { execFile } = require('child_process');
const { promisify } = require('util');

const execFileAsync = promisify(execFile);

const SAMPLE = require('./data').basic[0];
assert.equal(SAMPLE.name, 'ENGLISH');

const wasmBinary = fs.readFileSync(path.join(__dirname, '..', 'wasm', 'dist', 'cld.web.wasm'));

// Runs the child asynchronously (not execFileSync): the second case below
// needs an HTTP server alive *in this same process* to answer the child's
// request, which a synchronous, event-loop-blocking child_process call
// would deadlock against.
function runChild(script) {
return execFileAsync(process.execPath, ['-e', script], { encoding: 'utf8', cwd: __dirname });
}

(async () => {
const withoutOverride = `
import('../wasm/browser-entry.js')
.then(m => m.detect(${JSON.stringify(SAMPLE.sample)}))
.then(() => { console.log('UNEXPECTED_SUCCESS'); process.exit(0); })
.catch(() => { console.log('EXPECTED_FAILURE'); process.exit(0); });
`;
let out1;
try {
out1 = (await runChild(withoutOverride)).stdout;
} catch (e) {
throw new Error(`child process crashed running detect() without an override: ${e.message}`);
}
assert.match(out1, /EXPECTED_FAILURE/, 'detect() should fail without a locateFile override (file:// is not fetchable in Node)');

const server = http.createServer((req, res) => res.end(wasmBinary));
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
const port = server.address().port;

try {
const withOverride = `
import('../wasm/browser-entry.js').then(async m => {
m.setWasmModuleOptions({ locateFile: () => 'http://127.0.0.1:${port}/custom-path/cld.web.wasm' });
const result = await m.detect(${JSON.stringify(SAMPLE.sample)});
console.log(JSON.stringify(result.languages[0]));
process.exit(0);
}).catch(e => { console.error(e.stack); process.exit(1); });
`;
const out2 = (await runChild(withOverride)).stdout;
const topLanguage = JSON.parse(out2.trim().split('\n').pop());
assert.equal(topLanguage.name, 'ENGLISH', 'detect() with a locateFile override should still detect correctly');
} finally {
server.close();
}

console.log('Browser entry setWasmModuleOptions()/locateFile override verified');
})();
14 changes: 12 additions & 2 deletions wasm/browser-entry.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,28 @@
// the native path at all -- it goes straight to the WASM backend.

import createCldModule from './dist/cld.web.mjs';
import meta from '../lib/metadata.json';
import meta from '../lib/metadata.json' with { type: 'json' };
import { createDetect } from '../lib/detect-shape.js';
import { wrapWasmModule } from '../lib/wasm-wrap.js';

let moduleOptions = null;
let modulePromise = null;
function loadBackend() {
if (!modulePromise) {
modulePromise = createCldModule().then(wrapWasmModule);
modulePromise = createCldModule(moduleOptions ?? {}).then(wrapWasmModule);
}
return modulePromise;
}

// Lets consumers override where cld.web.wasm is fetched from (e.g. when a
// bundler moves wasm assets to a different path/CDN than the JS glue
// expects by default). Must be called before the first detect() call --
// loadBackend() only reads moduleOptions the first time it instantiates
// the module.
export function setWasmModuleOptions(options) {
moduleOptions = options;
}

export const LANGUAGES = meta.LANGUAGES;
export const DETECTED_LANGUAGES = meta.DETECTED_LANGUAGES;
export const ENCODINGS = meta.ENCODINGS;
Expand Down
Loading