Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -166,17 +166,13 @@ build-mcode-runtime:

.PHONY: build-microsandbox-provider check-microsandbox-provider
build-microsandbox-provider:
@test "$$(go env GOOS)" = linux || { echo 'The microsandbox provider helper requires Linux' >&2; exit 1; }
@set -e; output="$${OAC_DEV_HOME:-$$HOME/.oac}/build/microsandbox-provider"; \
[[ "$$output" == /* ]] || { echo 'Provider output directory must be absolute' >&2; exit 1; }; \
mkdir -p "$$output"; \
cd services/core/tools/microsandbox-provider; \
GOWORK=off CGO_ENABLED=1 go build -mod=readonly -trimpath -o "$$output/oac-microsandbox-provider" .
python3 scripts/build-microsandbox-provider.py build "$${OAC_DEV_HOME:-$$HOME/.oac}/build/microsandbox-provider/oac-microsandbox-provider"

check-microsandbox-provider:
python3 scripts/build-microsandbox-provider.test.py
go test -mod=readonly ./services/core/internal/sandbox/microsandbox/... -count=1
@if [[ "$$(go env GOOS)" == linux ]]; then \
cd services/core/tools/microsandbox-provider && GOWORK=off CGO_ENABLED=1 go test -mod=readonly ./... -count=1; \
python3 scripts/build-microsandbox-provider.py test; \
else \
printf 'Skipping the Linux-only microsandbox SDK helper tests; the full Linux gate is required before release.\n'; \
fi
Expand Down
8 changes: 4 additions & 4 deletions docs/maintainers.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ make build-core-distribution
| `OAC_NATIVE_INSTALLER_BUILD_DIR` | Native installer catalog directory; see [Native installers](#native-installers) |
| `CORE_DISTRIBUTION_BUILD_DIR` | Output directory under `~/.oac`. Default: `~/.oac/build/core-distribution` |
| `CORE_DISTRIBUTION_BUILD_NETWORK` | Docker build network: `default`, `host` or `none` |
| `CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE` | Cached microsandbox release archive. Default: `~/.oac/cache/microsandbox-v0.7.2-linux-x86_64.tar.gz`, downloaded when missing |
| `CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE` | Cached microsandbox release archive. Default: `~/.oac/cache/microsandbox-v0.7.8-linux-x86_64.tar.gz`, downloaded when missing |
| `CORE_DISTRIBUTION_DATABASE_IMAGE` | PostgreSQL 16 image; the default is pinned by its linux/amd64 manifest digest |

The build reuses the Core, Web, Runtime, SDK and helper builders. The manifest records the commit and source tree, image config and OCI manifest digests, the Runtime OCI manifest digest, the microsandbox runtime and firmware hashes, and the size and SHA-256 of every Runtime and node artifact; native installers carry only their SHA-256 in the [catalog](#native-installers). Output is the control archive and its `.sha256`, the optional offline archive, and the versioned Runtime, node and native installer assets. Nothing is published. Rebuilding into a directory that already holds this commit's distribution is refused.
Expand Down Expand Up @@ -103,16 +103,16 @@ make build-e2b-provider

Docker builds the Linux helper for `GOARCH=amd64` (default) or `GOARCH=arm64` with the pinned CPython and Debian 12 image. The Python dependency closure, including PyInstaller, is hash-locked in `services/core/tools/e2b-provider/requirements.lock`; no E2B account key is needed. Set `E2B_PROVIDER_BUILD_DIR` for another output directory. The build is a pure function of the helper sources, `LICENSE` and the build script, so it is cached under `~/.oac/cache/e2b-provider/` by their hash and rebuilt only when they change. The output is `oac-e2b-provider-linux-<architecture>.tar.gz` with its `.sha256`; it extracts to `oac-e2b-provider/` with the executable, `_internal/`, `licenses/`, `requirements.lock` and `manifest.json`. The Core image uses that tree; the host needs a compatible glibc and CA certificates, not Python.

**microsandbox helper.** Linux only, with a C compiler:
**microsandbox helper.** Linux amd64 only, with a C compiler:

```sh
make build-microsandbox-provider
make check-microsandbox-provider
```

The helper is written to `~/.oac/build/microsandbox-provider/oac-microsandbox-provider`. Its separate Go module pins the microsandbox Go SDK v0.7.2 and embeds the matching FFI library; never build production with the SDK's `microsandbox_ffi_path` tag. Core itself stays a CGO-disabled build. The helper needs glibc and runs only on nodes.
The helper is written to `~/.oac/build/microsandbox-provider/oac-microsandbox-provider`. Its separate Go module pins the official microsandbox SDK source commit. Both commands and the distribution build use `scripts/build-microsandbox-provider.py`: it stages the module dependencies, verifies the matching official FFI release checksum, fills the SDK's empty release bundle and builds with that FFI embedded. The SDK source is unchanged and no vendored binary is committed. Use this entry point rather than invoking `go build` directly; never build production with the SDK's `microsandbox_ffi_path` tag. Core itself stays a CGO-disabled build. The helper needs glibc and runs only on nodes.

**microsandbox runtime.** The distribution uses the official [v0.7.2 release](https://github.com/superradcompany/microsandbox/releases/tag/v0.7.2) archive `microsandbox-linux-x86_64.tar.gz`, SHA256 `47c223e3ef5298abf05f47ed9f87981106e400d99bb3f1d042d4d6881346b18b` (`RUNTIME_ARCHIVE_SHA256` in `scripts/core-distribution-manifest.py`). The build verifies the checksum before extracting `msb` and `libkrunfw.so.5.6.1` and records both files' hashes. The helper checks those hashes on every call and never installs or upgrades them.
**microsandbox runtime.** The distribution uses the official [v0.7.8 release](https://github.com/superradcompany/microsandbox/releases/tag/v0.7.8) archive `microsandbox-linux-x86_64.tar.gz`, SHA256 `86f9f72dc3e639c7175bc07909b4b63ce412517c1ef8a2e1921171af5682fded` (`RUNTIME_ARCHIVE_SHA256` in `scripts/core-distribution-manifest.py`). The build verifies the checksum before extracting `msb` and `libkrunfw.so.5.6.1` and records both files' hashes. The helper checks those hashes on every call and never installs or upgrades them.

### Standalone Core builds

Expand Down
10 changes: 5 additions & 5 deletions docs/zh/maintainers.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "构建并发布 OpenAgentCore"
source: docs/maintainers.md
source_hash: 494618f4d605d96bfcb9b1413f8224722324e1037764cc5b38baca102a8f1e90
source_hash: f91505a6502e09cc13b5ece7cb4ba578f6d2861621d70a1f836f89dcf9299ea4
---

本指南面向负责构建和发布 OpenAgentCore 的维护者。要安装 Core 和 Web,请使用 [安装指南](getting-started/install.md)。安装器代码遵循的规则见 [部署](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/deploy/README.md) 和 [节点安装器](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/deploy/node/README.md);必需检查见 [CONTRIBUTING](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#required-checks)。
Expand Down Expand Up @@ -35,7 +35,7 @@ make build-core-distribution
| `OAC_NATIVE_INSTALLER_BUILD_DIR` | 原生安装器目录;请参阅[原生安装器](#native-installers) |
| `CORE_DISTRIBUTION_BUILD_DIR` | `~/.oac` 下的输出目录。默认值:`~/.oac/build/core-distribution` |
| `CORE_DISTRIBUTION_BUILD_NETWORK` | Docker 构建网络:`default`、`host` 或 `none` |
| `CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE` | 已缓存的 microsandbox 发布归档。默认值:`~/.oac/cache/microsandbox-v0.7.2-linux-x86_64.tar.gz`,缺失时下载 |
| `CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE` | 已缓存的 microsandbox 发布归档。默认值:`~/.oac/cache/microsandbox-v0.7.8-linux-x86_64.tar.gz`,缺失时下载 |
| `CORE_DISTRIBUTION_DATABASE_IMAGE` | PostgreSQL 16 镜像;默认值通过其 linux/amd64 清单摘要固定 |

构建过程会复用 Core、Web、Runtime、SDK 和辅助程序构建器。清单会记录提交和源代码树、镜像配置及 OCI 清单摘要、Runtime OCI 清单摘要、microsandbox 运行时和固件哈希,以及每个 Runtime 和节点构件的大小与 SHA-256;原生安装器在[目录](#native-installers)中仅记录其 SHA-256。输出包括控制归档及其 `.sha256`、可选的离线归档,以及带版本号的 Runtime、节点和原生安装器资源。此过程不会发布任何内容。如果目标目录中已包含此提交的分发包,重建会拒绝执行。
Expand Down Expand Up @@ -105,16 +105,16 @@ make build-e2b-provider

Docker 使用固定版本的 CPython 和 Debian 12 镜像按 `GOARCH=amd64`(默认)或 `GOARCH=arm64` 构建 Linux 辅助程序。Python 依赖闭包(including PyInstaller)在 `services/core/tools/e2b-provider/requirements.lock` 中按哈希锁定;不需要 E2B 账户密钥。要使用其他输出目录,请设置 `E2B_PROVIDER_BUILD_DIR`。构建结果完全由辅助程序源代码、`LICENSE` 和构建脚本决定,因此会按它们的哈希缓存在 `~/.oac/cache/e2b-provider/` 下,仅在它们变化时重新构建。输出为 `oac-e2b-provider-linux-<architecture>.tar.gz` 及其 `.sha256`;解压后会得到 `oac-e2b-provider/`,其中包含可执行文件、`_internal/`、`licenses/`、`requirements.lock` 和 `manifest.json`。Core 镜像使用该目录树;主机需要兼容的 glibc 和 CA 证书,而不需要 Python。

**microsandbox 辅助程序。** 仅支持 Linux,并且需要 C 编译器:
**microsandbox 辅助程序。** 仅支持 Linux amd64,并且需要 C 编译器:

```sh
make build-microsandbox-provider
make check-microsandbox-provider
```

该辅助程序会写入 `~/.oac/build/microsandbox-provider/oac-microsandbox-provider`。其独立的 Go 模块固定 microsandbox Go SDK v0.7.2,并嵌入匹配的 FFI 库;构建生产版本时,绝不能使用该 SDK 的 `microsandbox_ffi_path` 标签。Core 本身仍采用禁用 CGO 的构建。该辅助程序需要 glibc,并且只能在节点上运行。
该辅助程序会写入 `~/.oac/build/microsandbox-provider/oac-microsandbox-provider`。其独立的 Go 模块固定官方 microsandbox SDK 源码提交。上述两个命令与分发构建均使用 `scripts/build-microsandbox-provider.py`:它暂存模块依赖,验证匹配的官方 FFI 发行文件校验和,填充 SDK 的空发行包,并构建嵌入该 FFI 的程序。SDK 源码保持不变,仓库不提交 vendored 二进制文件。请使用此入口,而非直接调用 `go build`;构建生产版本时,绝不能使用该 SDK 的 `microsandbox_ffi_path` 标签。Core 本身仍采用禁用 CGO 的构建。该辅助程序需要 glibc,并且只能在节点上运行。

**microsandbox 运行时。** 分发包使用官方的 [v0.7.2 release](https://github.com/superradcompany/microsandbox/releases/tag/v0.7.2) 归档 `microsandbox-linux-x86_64.tar.gz`,SHA256 为 `47c223e3ef5298abf05f47ed9f87981106e400d99bb3f1d042d4d6881346b18b`(即 `scripts/core-distribution-manifest.py` 中的 `RUNTIME_ARCHIVE_SHA256`)。构建过程会先验证校验和,再解压 `msb` 和 `libkrunfw.so.5.6.1`,并记录这两个文件的哈希。辅助程序会在每次调用时检查这些哈希,并且绝不安装或升级它们。
**microsandbox 运行时。** 分发包使用官方的 [v0.7.8 release](https://github.com/superradcompany/microsandbox/releases/tag/v0.7.8) 归档 `microsandbox-linux-x86_64.tar.gz`,SHA256 为 `86f9f72dc3e639c7175bc07909b4b63ce412517c1ef8a2e1921171af5682fded`(即 `scripts/core-distribution-manifest.py` 中的 `RUNTIME_ARCHIVE_SHA256`)。构建过程会先验证校验和,再解压 `msb` 和 `libkrunfw.so.5.6.1`,并记录这两个文件的哈希。辅助程序会在每次调用时检查这些哈希,并且绝不安装或升级它们。

### 独立 Core 构建 {#standalone-core-builds}

Expand Down
10 changes: 3 additions & 7 deletions scripts/build-core-distribution.sh
Original file line number Diff line number Diff line change
Expand Up @@ -110,16 +110,12 @@ cp services/core/deploy/codex/seccomp.json "$bundle/runtime/"
cp LICENSE "$bundle/"

OAC_DEV_BUILD_REVISION="$revision" scripts/build-core-image-context.sh "$stage/core"
(
cd services/core/tools/microsandbox-provider
GOWORK=off CGO_ENABLED=1 go build -mod=readonly -trimpath \
-o "$stage/core/bin/oac-microsandbox-provider" .
)
msb_archive="${CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE:-$runtime_root/cache/microsandbox-v0.7.2-linux-x86_64.tar.gz}"
python3 scripts/build-microsandbox-provider.py build "$stage/core/bin/oac-microsandbox-provider"
msb_archive="${CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE:-$runtime_root/cache/microsandbox-v0.7.8-linux-x86_64.tar.gz}"
if [[ ! -f "$msb_archive" ]]; then
mkdir -p "$(dirname "$msb_archive")"
curl --fail --location --proto '=https' --tlsv1.2 \
https://github.com/superradcompany/microsandbox/releases/download/v0.7.2/microsandbox-linux-x86_64.tar.gz \
https://github.com/superradcompany/microsandbox/releases/download/v0.7.8/microsandbox-linux-x86_64.tar.gz \
--output "$stage/microsandbox.download"
python3 scripts/core-distribution-manifest.py extract-runtime "$stage/microsandbox.download" "$stage/core/microsandbox"
mv "$stage/microsandbox.download" "$msb_archive"
Expand Down
90 changes: 90 additions & 0 deletions scripts/build-microsandbox-provider.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""Build the provider with the official SDK source and matching embedded FFI."""

import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import tempfile

SDK_MODULE = "github.com/superradcompany/microsandbox/sdk/go"
FFI_SHA256 = "bc079888050a92d3652191ae8e18fba96e1ac66dc78bce4e5aaeb7eca9b04e25"
FFI_FILE = "libmicrosandbox_go_ffi-linux-amd64.so"


def verify_ffi(path):
with Path(path).open("rb") as stream:
digest = hashlib.sha256()
for block in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(block)
if digest.hexdigest() != FFI_SHA256:
raise ValueError("Official microsandbox FFI checksum mismatch")


def install_ffi(source, destination):
verify_ffi(source)
if destination.is_symlink() or not destination.is_file() or destination.stat().st_size != 0:
raise ValueError("Official SDK must contain an empty FFI release sentinel")
shutil.copyfile(source, destination)


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("operation", choices=("build", "test"))
parser.add_argument("output", nargs="?", type=Path)
args = parser.parse_args()
if (args.operation == "build") != (args.output is not None):
parser.error("build requires an absolute output path; test takes no output")
if args.output is not None and not args.output.is_absolute():
parser.error("output must be absolute")
root = Path(__file__).resolve().parents[1]
module = root / "services/core/tools/microsandbox-provider"
env = dict(os.environ, GOWORK="off", CGO_ENABLED="1")
def run(*command, cwd=module):
try:
return subprocess.check_output(command, cwd=cwd, env=env, text=True)
except subprocess.CalledProcessError as error:
print(error.output, end="")
raise
if run("go", "env", "GOOS", "GOARCH").splitlines() != ["linux", "amd64"]:
parser.error("the provider distribution requires Linux amd64")
run("go", "mod", "download", SDK_MODULE)
sdk = json.loads(run("go", "list", "-mod=readonly", "-m", "-json", SDK_MODULE))
if "Replace" in sdk:
raise ValueError("The SDK must be the pinned official module")
versions = re.findall(r'^const sdkVersion = "([0-9.]+)"$', (Path(sdk["Dir"]) / "setup.go").read_text(), re.M)
if len(versions) != 1:
raise ValueError("Official SDK release declaration changed")
version = versions[0]
cache = Path(os.environ.get("OAC_DEV_HOME", str(Path.home() / ".oac"))) / "cache"
if not cache.is_absolute():
raise ValueError("OAC_DEV_HOME must be absolute")
cache.mkdir(parents=True, exist_ok=True)
ffi = cache / (FFI_SHA256 + ".so")
with tempfile.TemporaryDirectory(prefix="oac-microsandbox-build-", dir=cache) as temporary:
stage = Path(temporary)
if not ffi.exists():
download = stage / FFI_FILE
run("curl", "--fail", "--location", "--silent", "--show-error", "--retry", "3", "--output", str(download),
f"https://github.com/superradcompany/microsandbox/releases/download/v{version}/{FFI_FILE}")
verify_ffi(download)
os.replace(download, ffi)
verify_ffi(ffi)
for source in [*module.glob("*.go"), module / "go.mod", module / "go.sum"]:
shutil.copyfile(source, stage / source.name)
run("go", "mod", "vendor", "-o", str(stage / "vendor"))
install_ffi(ffi, stage / "vendor" / SDK_MODULE / "internal/bundle/bundles" / FFI_FILE)
if args.operation == "test":
print(run("go", "test", "-mod=vendor", "./...", "-count=1", cwd=stage), end="")
print(run("go", "vet", "-mod=vendor", "./...", cwd=stage), end="")
else:
args.output.parent.mkdir(parents=True, exist_ok=True)
print(run("go", "build", "-mod=vendor", "-trimpath", "-o", str(args.output), ".", cwd=stage), end="")


if __name__ == "__main__":
main()
48 changes: 48 additions & 0 deletions scripts/build-microsandbox-provider.test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/usr/bin/env python3
import hashlib
import importlib.util
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch

spec = importlib.util.spec_from_file_location("build_provider", Path(__file__).with_name("build-microsandbox-provider.py"))
build = importlib.util.module_from_spec(spec)
spec.loader.exec_module(build)


class OfficialBundleTests(unittest.TestCase):
def test_only_verified_payload_replaces_empty_sentinel(self):
with tempfile.TemporaryDirectory() as directory:
source, destination = Path(directory) / "ffi", Path(directory) / "sentinel"
source.write_bytes(b"official ffi")
destination.touch()
with patch.object(build, "FFI_SHA256", hashlib.sha256(source.read_bytes()).hexdigest()):
build.install_ffi(source, destination)
self.assertEqual(destination.read_bytes(), source.read_bytes())
with self.assertRaisesRegex(ValueError, "empty FFI"):
build.install_ffi(source, destination)

def test_corrupt_payload_does_not_replace_sentinel(self):
with tempfile.TemporaryDirectory() as directory:
source, destination = Path(directory) / "ffi", Path(directory) / "sentinel"
source.write_bytes(b"wrong release")
destination.touch()
with self.assertRaisesRegex(ValueError, "checksum mismatch"):
build.install_ffi(source, destination)
self.assertEqual(destination.read_bytes(), b"")

def test_symlink_sentinel_is_rejected(self):
with tempfile.TemporaryDirectory() as directory:
source, destination, outside = (Path(directory) / name for name in ("ffi", "sentinel", "outside"))
source.write_bytes(b"official ffi")
outside.touch()
destination.symlink_to(outside)
with patch.object(build, "FFI_SHA256", hashlib.sha256(source.read_bytes()).hexdigest()):
with self.assertRaisesRegex(ValueError, "empty FFI"):
build.install_ffi(source, destination)
self.assertEqual(outside.read_bytes(), b"")


if __name__ == "__main__":
unittest.main()
1 change: 1 addition & 0 deletions scripts/ci_plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
(("scripts/build-native-", "scripts/native-"), SCRIPTS, ("native", "backend", "distribution")),
(("scripts/build-core.sh", "scripts/build-core-image-context.sh"), (".sh",), ("backend", "api", "distribution", "native")),
(("deploy/distribution/",), ("Dockerfile",), ("backend", "api", "distribution", "native", "compose")),
(("scripts/build-microsandbox-provider.",), (".py",), ("backend", "distribution")),
(("scripts/build-e2b-provider.sh",), (".sh",), ("backend", "api", "distribution")),
(("scripts/build-claude", "scripts/check-claude", "scripts/build-mcode", "scripts/prepare-release-runtimes.sh"),
SCRIPTS, ("harness", "native", "backend", "distribution")),
Expand Down
Loading
Loading