Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Build checks hash this upstream snapshot byte-for-byte and compare generated
# CSS; preserve their canonical LF representation in Windows worktrees too.
web/design-system/tokens.json text eol=lf
web/src/styles/design-tokens.generated.css text eol=lf
# CLI contract snapshots also compare exact bytes.
server/cmd/mem/testdata/*.golden.json text eol=lf
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ jobs:
timeout-minutes: 30
env:
MEM_TEST_DB: postgres://mem:mem@localhost:5432/mem_test?sslmode=disable
MEM_OAUTH_TEST_DB: postgres://mem:mem@localhost:5432/mem_oauth_test?sslmode=disable
services:
postgres:
image: pgvector/pgvector:pg16@sha256:00ba258a66dac104fd5171074a0084462a64a1369d8513f3d0a634e2f24d15bc
Expand Down Expand Up @@ -126,6 +127,9 @@ jobs:
"${MEM_TEST_DB}"
up

- name: Create isolated GitHub OAuth acceptance database
run: psql "${MEM_TEST_DB}" -v ON_ERROR_STOP=1 -c 'CREATE DATABASE mem_oauth_test;'

- name: Test with race detector and coverage
working-directory: server
run: |
Expand Down
63 changes: 63 additions & 0 deletions .github/workflows/preview-container-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Build Sealos preview images

on:
push:
branches: ["feat/65-github-sealos", "preview/sealos-*"]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: mem-preview-images-${{ github.ref }}
cancel-in-progress: true

jobs:
images:
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
packages: write
strategy:
fail-fast: false
max-parallel: 1
matrix:
include:
- image: mem-preview-server
context: server
- image: mem-preview-web
context: web
- image: mem-preview-worker
context: worker
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and publish commit image
id: build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7
with:
context: ${{ matrix.context }}
platforms: linux/amd64
push: true
tags: ghcr.io/bytefolk/${{ matrix.image }}:${{ github.sha }}
labels: |
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
build-args: |
VERSION=sealos-preview
REVISION=${{ github.sha }}
WORKER_EXTRAS=
cache-from: type=gha,scope=mem-preview-${{ matrix.context }}
cache-to: type=gha,scope=mem-preview-${{ matrix.context }},mode=max
- name: Record image digest
env:
IMAGE_NAME: ${{ matrix.image }}
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
run: |
printf '%s\n' "ghcr.io/bytefolk/$IMAGE_NAME@$IMAGE_DIGEST" >> "$GITHUB_STEP_SUMMARY"
29 changes: 29 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ The project publishes 0.x prerelease versions; a stable release line is not yet

## [Unreleased]

### Personal GitHub preview

- Add allowlisted GitHub browser sign-in and explicit account binding with
PKCE, single-use browser-bound state, immutable provider subjects, verified
primary email and no automatic email linking. Browser cookies have
server-side expiry/revocation and mutation CSRF/origin checks. Existing
Bearer clients retain their contract.
- Add a small Sealos Helm overlay with closed password enrollment and explicit
allowlisted first-owner GitHub bootstrap. No model Worker is needed for
files and lexical memory recall. Real provider/deployment certification is
separate from the fixture tests.

### Security

- Refresh Web development-only transitive dependencies to patched
`brace-expansion` 1.1.21/2.1.7 and `undici` 7.30.0, restoring the unchanged
Linux and Windows real-registry audit gates. Production dependency ranges
and audit severity thresholds are unchanged.

### Added

- Per-agent memory namespace filter on lexical recall (`#227`): `POST /v1/context`
Expand All @@ -19,6 +38,16 @@ The project publishes 0.x prerelease versions; a stable release line is not yet

### Changed

- Correct native Sealos startup to use the compiled `local-fast-v2` profile,
a private 0700 transfer subdirectory and Worker probe timeouts that accommodate
Python/gRPC startup under the preview CPU limit.
- Render Sealos Web nginx configuration at startup from the shared image
template into its writable `/tmp` claim, avoiding unresolved instance
expressions in ConfigMap block scalars while retaining nonroot/read-only
container security and OAuth callback access-log filtering.
- Restore the Agent-memory integration gates for migration 0027 and build the
disposable MinIO/mc test dependency from pinned official source commits
when the upstream community image registries reject pulls.
- The live recall producer (`python3 -m benchmarks.recall produce`) reads engine, profile, provider and embedding dimension from the running memd instead of CLI labels, scores the multilingual v1 set (English and Chinese, including image-description queries), and stays on-demand rather than a CI gate. Refs #175.
- Rewrite the root README around a concise product promise, a verified Compose first-run path, source-built MCP setup, current capability and trust boundaries, and links to authoritative reference docs. Remove the unsupported unpublished `@bytefolk/mem-mcp@0.1.2` install path and keep its migration explicitly tracked in #153. Refs #229.
- Index generation HTTP create/activate/rollback stay `503 execution_unavailable` with `execution_wired: false` on that body, and the same flag is now on events as well as list/status/cancel/resume/discard. Create still returns `400` for malformed JSON or an empty `profile_id` before the 503. Empty-workspace activate is refused by the same 503. Refs #174.
Expand Down
12 changes: 12 additions & 0 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@

## 0. TL;DR

Personal GitHub browser login is optional and requires an exact public HTTPS
origin, OAuth client credentials and numeric GitHub subject allowlist. New
GitHub first-owner bootstrap is explicit and requires disabled public password
registration. HTTP surfaces: `GET /v1/auth/capabilities`,
`POST /v1/auth/github/start`, `GET /v1/auth/github/callback`,
`GET /v1/auth/session`, `POST /v1/auth/logout`, and authenticated
`GET /v1/auth/github/identity`. GitHub identities never link by email alone;
binding requires password reauthentication or a recent browser session and
the callback retains the same active actor/session. See
[`deploy/sealos/README.md`](deploy/sealos/README.md). This personal flow does
not claim the full hosted identity/OIDC/MFA roadmap is implemented.

**mem** 是一个开源、自托管、面向 Agent 的可迁移网盘。
形态像网盘,机器入口像共享 Memory Plane。

Expand Down
7 changes: 7 additions & 0 deletions deploy/helm/mem/secret.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,10 @@ MEM_S3_SECRET_KEY=replace-me
MEM_S3_USE_SSL=true
MEM_WORKER_AUTH_KEY_ID=memd-primary
MEM_WORKER_AUTH_KEY_B64=replace-with-standard-base64-for-exactly-32-random-bytes
# Optional personal GitHub login. Use the actual ingress origin and configure
# the OAuth App callback as <MEM_PUBLIC_URL>/v1/auth/github/callback.
# MEM_PUBLIC_URL=https://mem.example.com
# MEM_GITHUB_CLIENT_ID=replace-me
# MEM_GITHUB_CLIENT_SECRET=replace-me
# MEM_GITHUB_ALLOWED_USER_IDS=47820304
# MEM_GITHUB_BOOTSTRAP=true # requires runtime.registrationMode=disabled
125 changes: 125 additions & 0 deletions deploy/sealos/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Personal mem preview on Sealos

`template.yaml` is the native Sealos Template API recipe: one Web, one `memd`,
one small CPU/model-free Worker, private pgvector PostgreSQL 16, private Redis,
and a Sealos-managed private object bucket. Files, lexical memory recall,
checkpoints, and workspace transfer work without a model provider. The Worker
is authenticated but no Ollama server, ASR, face or CLIP model is bundled;
vector/image enrichment needs separately configured providers.

This preview reserves limits of 2.4 CPU, 2048Mi memory and 7Gi RWO storage
(plus managed object storage). Requests follow the Sealos quota ladder.
The actual namespace dry-run, balance and quota decide whether it fits.
Each process that writes uses a StatefulSet claim instead of unsupported
`emptyDir`; PostgreSQL uses a subdirectory so a fresh mounted volume initializes.

## Inputs

1. A Sealos namespace with sufficient quota and a registry accepting the built
`server/`, `web/` and model-free `worker/` images for linux/amd64. Build the
worktree revision, not a stale release. The optional preview workflow
publishes commit-SHA tags under `ghcr.io/bytefolk/mem-preview-{server,web,worker}`;
a workflow file is not evidence that the images were built or can be pulled.
2. The template's `pgvector/pgvector:0.8.1-pg16` image supplies the `vector`,
`pgcrypto` and `pg_trgm` extensions. The private Redis URL uses generated
credentials. Bucket keys/endpoint come from Sealos `object-storage-key`,
and the bucket name from its managed per-bucket Secret. No MinIO console or
database port is exposed. mem's MinIO client uses path-style S3 addressing.
3. The actual HTTPS hostname and Sealos ingress/TLS configuration.
4. A GitHub OAuth App owned by the operator, with callback URL exactly
`https://<actual-mem-host>/v1/auth/github/callback`. It needs only
`read:user user:email`; no repository or organization permission is used.

Supply the required Template inputs from a protected arguments file outside
Git. Generate the Worker auth key as standard Base64 of exactly 32 random
bytes. OAuth secrets and Worker keys enter the runtime Secret only; both
Docker build contexts exclude `.env*`. Do not print a rendered Secret or
commit arguments files. `secret.env.example` lists equivalent Helm keys.
OAuth identities, challenges and sessions are excluded from workspace bundles.

`MEM_REGISTRATION_MODE=disabled` closes password signup. The explicit
`MEM_GITHUB_BOOTSTRAP=true` switch permits only an allowlisted GitHub subject
to create the first owner atomically. Numeric ID `47820304` is the requested
personal account; usernames/email addresses are not authorization keys.
Further users cannot enroll. Existing email accounts must authenticate and
explicitly link GitHub; matching email never merges accounts.

## Rollout

Select the intended instance name and `defaults.app_host`, then read the actual
rendered Ingress hostname before finalizing the GitHub OAuth App callback.
The console hostname does not establish `SEALOS_CLOUD_DOMAIN`; the personal
Hangzhou workspace rendered `mem-peterguy326-f5396ac8.sealoshzh.site`.
The generic template defaults remain random to avoid duplicate instances.
Use public images only after verifying
anonymous pull. For private images, precreate a fixed pull Secret and add its
name to all application workloads/init containers; do not reference a missing
Secret or copy a broad personal GitHub token to the cluster.

The operator uses the Sealos Template API CLI with the protected input file,
performs its free dry-run/quota preview, then deploys exactly that template
instance. Every resource carries `cloud.sealos.io/deploy-on-sealos` and the
last App CR opens the HTTPS Web entry. A successful API response still needs
pod readiness, URL checks and real GitHub callback/asset acceptance.

Web renders the image's shared nginx template using only `MEMD_UPSTREAM` and
`MEM_MAX_BODY_SIZE`. The upstream arrives as a single-line Pod environment
value because the Template API leaves instance expressions inside ConfigMap
block scalars unresolved. Generated configs use the `/tmp` claim; the image's
main nginx config is copied there with its include redirected to the generated
server config. UID/GID 101, read-only root filesystem and dropped capabilities
are retained. nginx variables and OAuth callback access-log filtering remain
in the shared template.

`mem-migrate` runs as the single memd replica's init container, then memd
starts with `MEM_AUTO_MIGRATE=false`. Database-not-ready failures retry with
the Pod; do not bypass migration. This revision adds migration 0027. It
preserves existing users/memberships/machine tokens; downgrade refuses to
orphan a GitHub-only account. Back up an existing deployment first.

The native profile is `local-fast-v2`. Workspace transfers use
`/tmp/mem-workspace-transfers`; memd creates this private 0700 directory under
the writable claim and rejects a shared temporary root. Worker exec health
checks need time for Python imports and two gRPC waits: startup permits up to
180 seconds, with 15-second startup and 10-second steady-state probe timeouts.
The existing CPU, memory, authentication and container security limits remain.

The native recipe disables ingress access logging, and the inner nginx filters
OAuth callback query strings. It intentionally has no placeholder NetworkPolicy:
the base Helm chart's example CIDR would block Sealos dependencies/GitHub.
Before broader sharing, use verified namespace/pod labels and real egress
rules; do not mistake absence of a NetworkPolicy for network isolation.

For operators already running the Helm chart with provisioned dependencies,
`preview-values.yaml` remains a separate model-free/no-Worker overlay:

```sh
helm upgrade --install mem deploy/helm/mem \
--namespace "$MEM_NAMESPACE" \
-f deploy/sealos/preview-values.yaml \
--set-string images.server.repository="$MEM_SERVER_IMAGE_REPOSITORY" \
--set-string images.server.tag="$MEM_IMAGE_TAG" \
--set-string images.web.repository="$MEM_WEB_IMAGE_REPOSITORY" \
--set-string images.web.tag="$MEM_IMAGE_TAG" \
--set-string ingress.host="$MEM_PUBLIC_HOST" \
--set-string ingress.tls.secretName="$MEM_TLS_SECRET" \
--wait --timeout 10m
```

Create the chart's `mem-runtime` Secret from the equivalent runtime keys.
Its pre-install/pre-upgrade migration Job must finish before the rollout.

## Acceptance

- Web `/healthz` and internal `memd /readyz` succeed.
- Login offers GitHub and no public account-creation option.
- An anonymous password registration fails; an unlisted GitHub user fails.
- The allowlisted account creates the first workspace and can upload/download
a file and write/recall a sourced memory.
- Account menu shows the linked GitHub identity.
- Cookie-authenticated mutations require the exact origin and CSRF header.
- Logout revokes the server session; stale binding callbacks cannot log a user
back in or change accounts. OAuth callback query strings are not access-logged.

Unit/HTTP fixtures do not certify real GitHub or Sealos interoperability.
Retain the actual callback/login/asset acceptance result after the rollout.
Loading
Loading