feat(mcp): streamable HTTP transport for hosts that cannot install the engram binary

[jagoqui]

🔍 Problem Description

Engram's MCP server only speaks stdio, so every machine that uses it must install the engram binary. Some team machines cannot install binaries at all (locked-down corporate laptops, ephemeral dev containers, shared hosts), which leaves those users without Engram memory even though their MCP clients (VS Code, Claude Code, Cursor, OpenCode) already support remote MCP servers over Streamable HTTP.

This is a different case from #112. That issue was resolved by fixing the binary path for OpenCode headless mode: stdio was still viable because the binary was installed. Here there is no binary on the client host, so stdio cannot work.

💡 Proposed Solution

An opt-in Streamable HTTP transport (MCP spec 2025-03-26) that a user can run in a container, on their own machine or on a personal server, and consume with a standard remote-MCP config:

{
  "mcpServers": {
    "engram-remote": {
      "type": "http",
      "url": "${env:ENGRAM_REMOTE_URL}",
      "headers": {
        "Authorization": "Bearer ${env:ENGRAM_REMOTE_TOKEN}",
        "X-Engram-Subproject": "${env:ENGRAM_SUBPROJECT}"
      }
    }
  }
}

Scope (personal / single-user deployments, not a multi-tenant hosted MCP):

• engram mcp --transport=http serves /mcp (Streamable HTTP via mark3labs/mcp-go, already a dependency) and /health. stdio stays the default and is unchanged.
• The project comes from the X-Engram-Subproject header on every request; HTTP mode never falls back to the server's cwd.
• Local-only mode: optional ENGRAM_MCP_HTTP_TOKEN bearer guard, Origin/Host validation against DNS rebinding, loopback bind by default.
• Local-only mode: optional ENGRAM_MCP_HTTP_TOKEN bearer guard, Origin/Host validation against DNS rebinding, loopback bind by default.
• Optional Engram Cloud sync: the request bearer is the user's Engram Cloud token. It is validated through a new authenticated GET /auth/whoami route and bound to the owner account; a token from another account is rejected. ENGRAM_CLOUD_TOKEN is optional (bearer-only mode). Cloud traffic stays HTTPS-only, reusing the existing rule that bearers never travel over plaintext HTTP.
• Deployment: docker/http/Dockerfile + docker-compose.http.yml, env-file driven (docker/http/env.example), plus an optional Caddy tls profile for docker-compose.cloud.yml so local and server setups both use HTTPS. Existing docker-compose.cloud.yml defaults and the published cloud image are unchanged.
• An existing engram data dir can be mounted as the container's data volume to continue from local memories.

I have a working implementation on a branch (tests, docs, and an end-to-end run: VS Code → engram-http → container SQLite → HTTPS sync → Engram Cloud dashboard). Happy to split it into chained PRs once this is approved.

📦 Affected Area

MCP Server (tools, transport)

🔄 Alternatives Considered

• Installing the binary everywhere: not possible on the target machines.
• Exposing the existing REST API (engram serve): it is not MCP, so MCP clients cannot use it.
• Implementing MCP tools directly on the cloud server (Postgres): the cloud only stores sync chunks/mutations, so it would need a second search/store implementation with different semantics than FTS5.
• The deprecated HTTP+SSE transport (2024-11-05): superseded by Streamable HTTP.

📎 Additional Context

• Related precedent: Support HTTP/SSE MCP transport for headless OpenCode serve mode #112 (closed; stdio remained viable in that scenario).
• Security notes: loopback by default; bearer bound to the cloud principal via /auth/whoami; fail-closed on cloud errors; startup refuses an http:// cloud URL in cloud mode; warnings when the endpoint can be reached without a secret.