Skip to content

feat(cloud): add managed project API (POST /admin/projects) and local deployment tooling聽#1582

Description

@gonzalez962

馃攳 Problem Description

In self-hosted Engram Cloud deployments adopting the modern managed-principals architecture (PrincipalSourceManagedToken, HumanUser, cloud_project_grants), there is currently no API to formally register or create a new project at the system level:

  1. Static and isolated legacy allowlist: The only system-level project allowlist mechanism is the legacy environment variable ENGRAM_CLOUD_ALLOWED_PROJECTS. This variable applies only to the single static ENGRAM_CLOUD_TOKEN env-token authorizer, requires container/server restarts to change, and is bypassed by managed principals during sync authorization.
  2. Missing system-level project controls: Managed principals authorize project sync via per-user records in cloud_project_grants (AuthorizeProjectForPrincipal). While administrators can assign grants to users for arbitrary project strings via POST /admin/users/{id}/grants or the dashboard, no corresponding project entity is created in cloud_project_controls. This leaves projects without system-level registration, prevents global sync pause/resume controls from reflecting them, and lacks an audit trail for project creation.
  3. No self-service project creation API: An operator or automation cannot programmatically create a project and receive the necessary administrative grant in a single operation.
  4. Local testing friction: Validating managed cloud workflows locally currently requires either standing up an external Dokploy/Coolify stack or running against pre-published GHCR images rather than the local working tree.

馃挕 Proposed Solution

Implement a dedicated managed-admin endpoint POST /admin/projects along with atomic PostgreSQL persistence, audit logging, and local deployment tooling:

  1. POST /admin/projects API Route:

    • Gated to managed admin principals (requireManagedAdmin).
    • Strict payload validation (createAdminProjectRequest with DisallowUnknownFields): {"name": "<project-name>"}.
    • Name canonicalization via cloudstore.NormalizeProjectGrant (trim, lowercase, collapse -- and __).
    • Atomic transaction via cloudstore.CreateProjectWithGrantAndAudit:
      • Inserts the project control record in cloud_project_controls (sync_enabled = true). Duplicate projects return ErrProjectAlreadyExists (mapped to HTTP 409 Conflict).
      • Auto-grants access to the acting admin in cloud_project_grants.
      • Records a project.create audit event in cloud_auth_audit_log (rolling back the entire transaction if audit logging fails or contains sensitive keys).
    • Response: 201 Created with {"name": "<normalized>", "sync_enabled": true}.
  2. Local Production-like Stack:

    • Add docker-compose.prod-local.yml and .env.example using isolated loopback ports (127.0.0.1:35432 Postgres, 127.0.0.1:38080 Cloud) building directly from docker/cloud/Dockerfile to test the exact working tree without port collisions with existing compose stacks.
    • Documentation in docs/engram-cloud/prod-local.md covering secret generation, stack lifecycle, CLI bootstrap, and endpoint usage.
  3. Executable Bruno API Collection:

    • Provide Bruno OpenCollection in docs/bruno/engram-cloud-self-hosted/ with requests for POST /admin/projects (Admin/11-create-project.yml), user management, and cloud sync.

馃摝 Affected Area

Other

馃攧 Alternatives Considered

  • Editing ENGRAM_CLOUD_ALLOWED_PROJECTS: Only works for the legacy single env-token sync; does not provision managed grants or cloud_project_controls, and requires server restarts.
  • Grant-only creation via POST /admin/users/{id}/grants: Assigns access to a specific user, but does not initialize cloud_project_controls or record a project.create audit log.

馃搸 Additional Context

  • Implemented and verified on branch feat/cloud-project-api (36 files including Go handlers, store transactions, unit/integration tests, Docker Compose, docs, and Bruno specs).
  • Complete test coverage in internal/cloud/cloudserver/admin_handlers_test.go (TestAdminCreateProject*) and internal/cloud/cloudstore/identity_storage_test.go (TestCreateProjectWithGrantAndAuditLifecycle).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions