Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/wrapper-maven-dependency.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"rumble-lsp-wrapper": patch
"jsoniq-language-server": patch
---

Build the wrapper jar with RumbleDB as a regular Maven dependency. The jar now contains a single version of each library: it used to mix classes from two versions of some, such as `jackson-core` 2.18.2 and 2.18.6.
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
java-version: "17"
cache: maven

- name: Cache assembled RumbleDB JAR
- name: Cache built RumbleDB JAR
uses: actions/cache@v5
with:
path: packages/rumble-lsp-wrapper/.cache/rumble
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
java-version: "17"
cache: maven

- name: Cache assembled RumbleDB JAR
- name: Cache built RumbleDB JAR
uses: actions/cache@v5
with:
path: packages/rumble-lsp-wrapper/.cache/rumble
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
java-version: "17"
cache: maven

- name: Cache assembled RumbleDB JAR
- name: Cache built RumbleDB JAR
uses: actions/cache@v5
with:
path: packages/rumble-lsp-wrapper/.cache/rumble
Expand Down
70 changes: 70 additions & 0 deletions .github/workflows/wrapper-prod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: Wrapper release jar

# Test CI only builds the wrapper for development, against Maven's classpath.
# This builds the shaded release jar and runs the wrapper and language server tests against it,
# so packaging problems show up before a release.
on:
push:
branches:
- main
paths:
- "packages/rumble-lsp-wrapper/**"
- "packages/language-server/src/integrations/rumble/**"
- ".github/workflows/wrapper-prod.yml"
pull_request:
paths:
- "packages/rumble-lsp-wrapper/**"
- "packages/language-server/src/integrations/rumble/**"
- ".github/workflows/wrapper-prod.yml"

env:
HUSKY: 0

jobs:
release-jar:
runs-on: ubuntu-latest

steps:
- name: Check out repository
uses: actions/checkout@v6

- name: Set up pnpm
uses: pnpm/action-setup@v6

- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 26
cache: pnpm

- name: Set up Java
uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "17"
cache: maven

- name: Cache built RumbleDB JAR
uses: actions/cache@v5
with:
path: packages/rumble-lsp-wrapper/.cache/rumble
key: rumbledb-${{ runner.os }}-java17-${{ hashFiles('packages/rumble-lsp-wrapper/scripts/fetch-rumble.sh') }}

- name: Install dependencies
run: pnpm install --frozen-lockfile

# The prod profile runs the wrapper's tests against the shaded jar.
- name: Build and test release jar
run: pnpm --filter rumble-lsp-wrapper run build:prod

- name: Generate grammar
run: pnpm run generate:grammar

- name: Build language server assets
run: pnpm --filter jsoniq-language-server run build:assets

# The language server starts the release jar with -jar, as it does for users.
- name: Test language server with release jar
run: |
export JSONIQ_LSP_WRAPPER_JAR="$(ls "$PWD"/packages/rumble-lsp-wrapper/target/rumble-lsp-wrapper-*-all.jar)"
pnpm run test:server
2 changes: 1 addition & 1 deletion packages/language-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Files are parsed with ANTLR grammars for both languages, which keeps these featu

Type-checking diagnostics, inferred types and running queries come from RumbleDB, through the [Java wrapper](https://github.com/RumbleDB/language-tools/tree/main/packages/rumble-lsp-wrapper). This needs Java 17 or newer on your `PATH`.

The wrapper JAR is too large to ship in the npm package. The server downloads it from GitHub Releases when it is first needed, checks its SHA-256 against the bundled manifest, and caches it. Set `JSONIQ_LSP_CACHE_DIR` to control where it is stored.
The wrapper JAR is too large to ship in the npm package. The server downloads it from GitHub Releases when it is first needed, checks its SHA-256 against the bundled manifest, and caches it. Set `JSONIQ_LSP_CACHE_DIR` to control where it is stored. To use another wrapper JAR, such as one you built yourself, set `JSONIQ_LSP_WRAPPER_JAR` to its path.

If you don't want any of this, set `wrapper.enabled` to `false` (see below) and the server runs on the parser alone.

Expand Down
21 changes: 5 additions & 16 deletions packages/language-server/src/integrations/rumble/executable/dev.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,6 @@ import { WrapperLaunchConfig } from "./index.js";
const PACKAGE_ROOT = findPackageRoot();
const WRAPPER_DEVELOPMENT_FOLDER = path.join(PACKAGE_ROOT, "../rumble-lsp-wrapper");
const WRAPPER_JAR_DEVELOPMENT_FOLDER = path.join(WRAPPER_DEVELOPMENT_FOLDER, "target");
const RUMBLE_JAR_DEVELOPMENT_PATH = path.join(
WRAPPER_DEVELOPMENT_FOLDER,
"generated-resources",
"rumbledb-current-jar-with-dependencies.jar",
);
const WRAPPER_PACKAGE_JSON_PATH = path.join(WRAPPER_DEVELOPMENT_FOLDER, "package.json");

const WRAPPER_RUNTIME_CLASSPATH_FILE = "runtime-classpath.txt";
Expand All @@ -22,21 +17,15 @@ const WRAPPER_MAIN_CLASS = "org.jsoniq.lsp.wrapper.Main";
export function resolveDevLaunchConfig(): WrapperLaunchConfig | undefined {
const localJarPath = resolveWrapperJarPath();
const classpathPath = path.join(WRAPPER_JAR_DEVELOPMENT_FOLDER, WRAPPER_RUNTIME_CLASSPATH_FILE);
if (
localJarPath === undefined ||
!fs.existsSync(classpathPath) ||
!fs.existsSync(RUMBLE_JAR_DEVELOPMENT_PATH)
) {
if (localJarPath === undefined || !fs.existsSync(classpathPath)) {
return undefined;
}

const runtimeClasspath = fs.readFileSync(classpathPath, "utf8").trim();
const classpathEntries = [
localJarPath,
RUMBLE_JAR_DEVELOPMENT_PATH,
...runtimeClasspath.split(path.delimiter),
].filter((entry): entry is string => entry !== undefined && entry.length > 0);
const classpath = [...new Set(classpathEntries)].join(path.delimiter);
// The runtime classpath includes RumbleDB and its dependencies from the local Maven repository.
const classpath = [localJarPath, ...runtimeClasspath.split(path.delimiter)]
.filter((entry) => entry.length > 0)
.join(path.delimiter);

return {
args: ["-cp", classpath, WRAPPER_MAIN_CLASS, "--daemon"],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { createLogger } from "server/utils/logger.js";
import { resolveDevLaunchConfig } from "./dev.js";
import { DownloadProgressReporter } from "./download.js";
import { resolveProdLaunchConfig } from "./prod.js";
import { SPARK_JVM_ARGS, sparkConfToJvmArgs } from "./utils.js";
import { resolveWrapperJarOverride, SPARK_JVM_ARGS, sparkConfToJvmArgs } from "./utils.js";

const logger = createLogger("wrapper:jar-resolution");

Expand All @@ -20,13 +20,7 @@ export interface WrapperResolutionOptions {
export async function resolveWrapperLaunchConfig(
options: WrapperResolutionOptions = {},
): Promise<WrapperLaunchConfig> {
const devConfig = resolveDevLaunchConfig();
const config = devConfig ?? (await resolveProdLaunchConfig(options));
if (devConfig === undefined) {
logger.debug(
"No development wrapper configuration found, falling back to production configuration.",
);
}
const config = await resolveLaunchConfig(options);

const sparkConfArgs = sparkConfToJvmArgs(buildSparkConf(serverConfig.wrapper), (key) =>
logger.warn(
Expand All @@ -38,3 +32,24 @@ export async function resolveWrapperLaunchConfig(
args: [...SPARK_JVM_ARGS, ...sparkConfArgs, ...config.args],
};
}

async function resolveLaunchConfig(
options: WrapperResolutionOptions,
): Promise<WrapperLaunchConfig> {
const overrideConfig = resolveWrapperJarOverride();
if (overrideConfig !== undefined) {
logger.info(
`Using the wrapper jar from JSONIQ_LSP_WRAPPER_JAR: ${process.env.JSONIQ_LSP_WRAPPER_JAR}`,
);
return overrideConfig;
}

const devConfig = resolveDevLaunchConfig();
if (devConfig !== undefined) {
return devConfig;
}
logger.debug(
"No development wrapper configuration found, falling back to production configuration.",
);
return resolveProdLaunchConfig(options);
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { createLogger } from "server/utils/logger.js";

import { createTerminalProgressReporter, downloadWithProgress } from "./download.js";
import { type WrapperLaunchConfig, type WrapperResolutionOptions } from "./index.js";
import { computeFileSha256, getWrapperCacheDirectory } from "./utils.js";
import { computeFileSha256, getWrapperCacheDirectory, jarLaunchConfig } from "./utils.js";

/// Production environment: use package assets instead of compiled output.
const PACKAGE_ROOT = findPackageRoot();
Expand Down Expand Up @@ -108,8 +108,5 @@ export async function resolveProductionJarPath(
export async function resolveProdLaunchConfig(
options: WrapperResolutionOptions = {},
): Promise<WrapperLaunchConfig> {
const cachedJarPath = await resolveProductionJarPath(options);
return {
args: ["-jar", cachedJarPath, "--daemon"],
};
return jarLaunchConfig(await resolveProductionJarPath(options));
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";

import { type WrapperLaunchConfig } from "./index.js";

const LANGUAGE_SERVER_CACHE_DIRNAME = "jsoniq-language-server";
const WRAPPER_CACHE_SUBDIRECTORY = "wrapper";

Expand Down Expand Up @@ -36,6 +38,28 @@ export function getBaseCacheDirectory(
return path.join(os.homedir(), ".cache");
}

/** Runs a shaded wrapper jar the way a release does. */
export function jarLaunchConfig(jarPath: string): WrapperLaunchConfig {
return { args: ["-jar", jarPath, "--daemon"] };
}

/**
* JSONIQ_LSP_WRAPPER_JAR runs the given wrapper jar instead of the development build or the
* released jar, for example to test a locally built release jar.
*/
export function resolveWrapperJarOverride(
env: NodeJS.ProcessEnv = process.env,
): WrapperLaunchConfig | undefined {
const jarPath = env.JSONIQ_LSP_WRAPPER_JAR;
if (jarPath === undefined || jarPath.length === 0) {
return undefined;
}
if (!fs.existsSync(jarPath)) {
throw new Error(`JSONIQ_LSP_WRAPPER_JAR is set to '${jarPath}', which does not exist.`);
}
return jarLaunchConfig(path.resolve(jarPath));
}

export function getWrapperCacheDirectory(
env: NodeJS.ProcessEnv = process.env,
platform: NodeJS.Platform = process.platform,
Expand Down
23 changes: 23 additions & 0 deletions packages/language-server/tests/wrapper/executable-utils.test.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,36 @@
import os from "node:os";
import path from "node:path";
import { fileURLToPath } from "node:url";

import {
getBaseCacheDirectory,
getWrapperCacheDirectory,
resolveWrapperJarOverride,
sparkConfToJvmArgs,
} from "server/integrations/rumble/executable/utils";
import { describe, expect, it } from "vitest";

describe("wrapper jar override", () => {
it("is not used when JSONIQ_LSP_WRAPPER_JAR is unset or empty", () => {
expect(resolveWrapperJarOverride({})).toBeUndefined();
expect(resolveWrapperJarOverride({ JSONIQ_LSP_WRAPPER_JAR: "" })).toBeUndefined();
});

it("runs the given jar like a release", () => {
const jarPath = fileURLToPath(import.meta.url);
expect(resolveWrapperJarOverride({ JSONIQ_LSP_WRAPPER_JAR: jarPath })).toEqual({
args: ["-jar", jarPath, "--daemon"],
});
});

it("fails when the jar does not exist", () => {
const jarPath = path.join(os.tmpdir(), "jsoniq-lsp-missing-wrapper.jar");
expect(() => resolveWrapperJarOverride({ JSONIQ_LSP_WRAPPER_JAR: jarPath })).toThrow(
jarPath,
);
});
});

describe("wrapper executable cache directories", () => {
it("uses JSONIQ_LSP_CACHE_DIR when explicitly provided", () => {
expect(
Expand Down
1 change: 1 addition & 0 deletions packages/rumble-lsp-wrapper/.gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
target/
/rumbledb
generated-resources/
.mvn/maven.config
4 changes: 3 additions & 1 deletion packages/rumble-lsp-wrapper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ pnpm run build:wrapper

This fetches RumbleDB at the commit pinned in `scripts/fetch-rumble.sh`, builds it, and then builds the wrapper. Both steps are cached: the wrapper is only rebuilt when its Java sources, Maven configuration, version or RumbleDB JAR change. To skip the cache, run `pnpm run build:force` in this package. Production builds are never cached.

Run the tests with `pnpm test` in this package, and format the Java code with `pnpm run fmt`.
Each RumbleDB build is installed into the local Maven repository under its own version, such as `3.1.0-local-660355a3f22b`, named after the jar's contents. Old versions stay in `~/.m2/repository/com/github/rumbledb/rumbledb/` and can be deleted.

Run the tests with `pnpm test` in this package, and format the Java code with `pnpm run fmt`. `pnpm run build:prod` runs the same tests against the shaded release jar.

When a local build exists, the language server uses it. Otherwise it falls back to the JAR from GitHub Releases.

Expand Down
Loading
Loading