feat(storage)!: storage authz completion

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+## [Unreleased]
+
+### ⚠ BREAKING CHANGES
+
+* **storage:** Client APIs no longer create missing containers (404). An omitted folder now resolves to a personal `cnd_<userId>/` folder. Creating under a missing or unmanaged `cnd_<otherUserId>/` path is denied. File create with `scope` now requires `edit` on that scope, not `read`. Client list-files is deferred.
+
+### Features
+
+* **storage:** complete filesystem-shaped ReBAC for Container, Folder, and File behind `authorization.enabled` ([#1173](https://github.com/ConduitPlatform/Conduit/issues/1173)). Leftover unowned folders/containers are unmanaged until the first write heals them; old files are not backfilled. There is no reconstruct job and no second filesystem flag. Provision named containers via Admin. Admin writes without `scope` make a folder container-owned (Client 403 on the default container); pass a scope if Client users should keep writing. Products that share prefixes (for example fyllo) should keep authorization off until they have per-user folder roots and per-file grants or privileged fetch.
+
 ## [0.17.0-alpha.6](https://github.com/ConduitPlatform/Conduit/compare/v0.17.0-alpha.5...v0.17.0-alpha.6) (2026-07-26)
 
 

modules/storage/README.md

@@ -0,0 +1,38 @@
+# Storage
+
+Authorization for containers, folders, and files. Client list-files is deferred.
+
+## Client breaking changes
+
+These apply to Storage Client routes and gRPC calls that use the user file handlers.
+
+- **Missing containers are not created.** Creating or updating a file with a container that does not already exist returns `404 Not Found`. `allowContainerCreation` still only affects Admin implicit container creation.
+- **Omitted folder becomes a personal folder.** If `folder` is omitted on Client file create, Storage uses `cnd_<userId>/`. Passing `/` still stores at the container root.
+- **Personal-folder squat is denied.** Creating a missing `cnd_<otherUserId>/` path returns `403 Permission Denied`. When `authorization.enabled` is true, an existing but **unmanaged** `cnd_<otherUserId>/` root is also denied. A user may create their own `cnd_<userId>/`. If another user's personal root already exists **and is managed**, normal folder edit checks apply.
+- **Scope create requires `edit`.** Creating a file with `scope` now requires `edit` on that scope, not `read`.
+
+Admin-only container create remains available. Public file reads without auth (`getFile` / `getFileUrl`), module schema ownership, and existing public URI / CDN / content-disposition / local URL upload behavior are unchanged.
+
+## Authorization tree
+
+When `authorization.enabled` is true, Storage registers `Container`, `Folder`, and `File` resources and maintains owner relations that follow the path. There is **no** second `authorization.filesystem.enabled` flag.
+
+- A container may own first-level folders, or files stored at `/`.
+- A folder owns nested folders and files.
+- Client file creates also stamp `scope ?? User:<id>` on the File so the creator can `can(File)` even if the folder has no owners yet.
+- The default container is never owned. It is created on both the database and the storage provider if missing.
+- An optional `scope` (for example `Team:<id>`) is attached as an extra owner when provided. Admin folder create without scope only attaches the container as the first-folder owner. Scope is optional and is not rejected when missing.
+
+**Upgrade / leftover data:** there is no reconstruct-indexes job and old files are not backfilled. A leftover folder or non-default container with no owner/editor/reader relations is **unmanaged**: folder/container `can(edit)` is skipped, and the first successful write heals it by attaching the current subject (plus Container/parent links). After that, normal `can(edit)` applies. Old private files without a File relation stay Client-inaccessible; Admin can still read/update/delete them.
+
+If Admin writes into a leftover (or new) folder **without** `scope`, the folder becomes container-owned. On the default container that means Client users will get `403` on later writes. That is expected. To keep the folder Client-writable, Admin must pass a `scope`, or let a Client user write first so they become the owner.
+
+Folder delete removes nested folders/files and all of their relations. Container delete pages those cleanups and also clears `Container` relations. File moves always try to drop the old structural Folder/Container owner (ignore missing) and add the new one.
+
+## Provisioning notes
+
+- Provision named containers via **Admin**. Client APIs will not create them.
+- For shared leftover folders that Client users should keep writing to, Admin should pass a `scope` (for example `Team:<id>`). Omitting scope is valid for Admin-only trees; it is not an error.
+- Do **not** enable `authorization.enabled` until the product has a folder ownership model **and** either per-file grants or a privileged fetch path.
+- Shared prefixes (`docs/`, team drops, fyllo-style common roots) become first-writer-wins on the first Client write after enable, then exclusive to that subject unless relations are granted.
+- Fyllo-like apps that share prefixes should keep `authorization.enabled: false` until they do that separate product work. This module does not migrate those apps.

modules/storage/package.json

@@ -30,7 +30,7 @@
     "prepare": "npm run build",
     "build:docker": "docker build -t ghcr.io/conduitplatform/storage:latest -f ./Dockerfile ../../ && docker push ghcr.io/conduitplatform/storage:latest",
     "generateTypes": "sh build.sh",
-    "test": "tsc -p tsconfig.test.json && node --test dist-test/adapter/StorageParamAdapter.test.js dist-test/migrations/fileUriMigration.test.js dist-test/providers/aws/acl.test.js dist-test/providers/google/folderMarkers.test.js dist-test/providers/google/deleteFolder.test.js dist-test/providers/google/publicAccess.test.js dist-test/utils/filePrivacy.test.js"
+    "test": "tsc -p tsconfig.test.json && node --test dist-test/adapter/StorageParamAdapter.test.js dist-test/migrations/fileUriMigration.test.js dist-test/providers/aws/acl.test.js dist-test/providers/google/folderMarkers.test.js dist-test/providers/google/deleteFolder.test.js dist-test/providers/google/publicAccess.test.js dist-test/utils/filePrivacy.test.js dist-test/__tests__/helpers.test.js dist-test/__tests__/relations.test.js dist-test/__tests__/folders.test.js dist-test/__tests__/cascade.test.js dist-test/__tests__/bootstrap.test.js dist-test/__tests__/file.authz.test.js"
   },
   "keywords": [],
   "author": "",

modules/storage/src/Storage.ts

@@ -50,7 +50,8 @@ import {
   type ImportResult,
 } from '@conduitplatform/module-tools';
 import { StorageParamAdapter } from './adapter/StorageParamAdapter.js';
-import { FileResource } from './authz/index.js';
+import { ContainerResource, FileResource, FolderResource } from './authz/index.js';
+import { ensureDefaultContainer } from './authz/bootstrap.js';
 import { AdminFileHandlers } from './admin/adminFile.js';
 import { randomBytes } from 'node:crypto';
 import { fileURLToPath } from 'node:url';
@@ -248,9 +249,11 @@ export default class Storage extends ManagedModule<Config> {
         this._storageAuthzResourceDispose?.();
         this._storageAuthzResourceDispose = this.grpcSdk.oncePeerUp(
           'authorization',
-          () => {
+          async () => {
             this._storageAuthzResourceDispose = null;
-            this.grpcSdk.authorization!.defineResource(FileResource);
+            await this.grpcSdk.authorization!.defineResource(ContainerResource);
+            await this.grpcSdk.authorization!.defineResource(FolderResource);
+            await this.grpcSdk.authorization!.defineResource(FileResource);
           },
         );
       } else {
@@ -269,6 +272,7 @@ export default class Storage extends ManagedModule<Config> {
       });
       this._fileHandlers.updateProvider(this.storageProvider);
       this._adminFileHandlers.updateProvider(this.storageProvider);
+      await ensureDefaultContainer(this.storageProvider);
       // Run the public container migration once after provider is configured
       if (!this.publicContainerMigrationRan && provider !== 'local') {
         this.publicContainerMigrationRan = true;

modules/storage/src/__tests__/bootstrap.test.ts

@@ -0,0 +1,71 @@
+import { afterEach, describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { ConfigController } from '@conduitplatform/module-tools';
+import { _StorageContainer } from '../models/index.js';
+import { ensureDefaultContainer } from '../authz/bootstrap.js';
+
+const originalConfig = ConfigController.getInstance().config;
+const originalGetInstance = _StorageContainer.getInstance.bind(_StorageContainer);
+
+afterEach(() => {
+  ConfigController.getInstance().config = originalConfig;
+  _StorageContainer.getInstance = originalGetInstance;
+});
+
+describe('ensureDefaultContainer', () => {
+  it('creates a missing default container in DB and on the provider without owning it', async () => {
+    ConfigController.getInstance().config = {
+      authorization: { enabled: true },
+      defaultContainer: 'conduit',
+    };
+    let created: { name: string; isPublic?: boolean } | undefined;
+    _StorageContainer.getInstance = (() => ({
+      findOne: async () => null,
+      create: async (doc: { name: string; isPublic?: boolean }) => {
+        created = doc;
+        return { _id: 'c1', ...doc };
+      },
+    })) as unknown as typeof _StorageContainer.getInstance;
+
+    const providerCalls: string[] = [];
+    const container = await ensureDefaultContainer({
+      containerExists: async (name: string) => {
+        providerCalls.push(`exists:${name}`);
+        return false;
+      },
+      createContainer: async (name: string) => {
+        providerCalls.push(`create:${name}`);
+        return true;
+      },
+    } as never);
+
+    assert.equal(container.name, 'conduit');
+    assert.equal(created?.isPublic, false);
+    assert.deepEqual(providerCalls, ['exists:conduit', 'create:conduit']);
+  });
+
+  it('is idempotent when the default container already exists on DB and provider', async () => {
+    ConfigController.getInstance().config = {
+      defaultContainer: 'conduit',
+    };
+    let createDb = 0;
+    _StorageContainer.getInstance = (() => ({
+      findOne: async () => ({ _id: 'c1', name: 'conduit' }),
+      create: async () => {
+        createDb += 1;
+        return { _id: 'c1', name: 'conduit' };
+      },
+    })) as unknown as typeof _StorageContainer.getInstance;
+
+    let createProvider = 0;
+    await ensureDefaultContainer({
+      containerExists: async () => true,
+      createContainer: async () => {
+        createProvider += 1;
+        return true;
+      },
+    } as never);
+    assert.equal(createDb, 0);
+    assert.equal(createProvider, 0);
+  });
+});

modules/storage/src/__tests__/cascade.test.ts

@@ -0,0 +1,229 @@
+import { afterEach, describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { ConfigController } from '@conduitplatform/module-tools';
+import { ConduitGrpcSdk } from '@conduitplatform/grpc-sdk';
+import { _StorageContainer, _StorageFolder, File } from '../models/index.js';
+import { folderPrefixRegex } from '../authz/helpers.js';
+import { deleteContainerTree, deleteFolderTree } from '../authz/cascade.js';
+
+const originalConfig = ConfigController.getInstance().config;
+const originalFileGetInstance = File.getInstance.bind(File);
+const originalFolderGetInstance = _StorageFolder.getInstance.bind(_StorageFolder);
+const originalContainerGetInstance =
+  _StorageContainer.getInstance.bind(_StorageContainer);
+
+afterEach(() => {
+  ConfigController.getInstance().config = originalConfig;
+  File.getInstance = originalFileGetInstance;
+  _StorageFolder.getInstance = originalFolderGetInstance;
+  _StorageContainer.getInstance = originalContainerGetInstance;
+});
+
+describe('folder delete prefix', () => {
+  it('escapes regex so foo.bar/ does not match fooXbar/', () => {
+    const prefix = folderPrefixRegex('foo.bar/');
+    const re = new RegExp(prefix.$regex);
+    assert.equal(re.test('foo.bar/'), true);
+    assert.equal(re.test('foo.bar/nested/'), true);
+    assert.equal(re.test('fooXbar/'), false);
+  });
+});
+
+describe('deleteFolderTree', () => {
+  it('deletes relations for every nested folder and file, then the provider and DB once', async () => {
+    ConfigController.getInstance().config = {
+      authorization: { enabled: true },
+      defaultContainer: 'conduit',
+    };
+    const deletedRelations: Array<{ subject?: string; resource?: string }> = [];
+    const deletedFolders: object[] = [];
+    const deletedFiles: object[] = [];
+    let providerDeletes = 0;
+
+    _StorageFolder.getInstance = (() => ({
+      findMany: async (_query: object, options?: { skip?: number; limit?: number }) => {
+        const all = [{ _id: 'dir1' }, { _id: 'dir2' }];
+        return all.slice(
+          options?.skip ?? 0,
+          (options?.skip ?? 0) + (options?.limit ?? 100),
+        );
+      },
+      deleteMany: async (query: object) => {
+        deletedFolders.push(query);
+      },
+    })) as unknown as typeof _StorageFolder.getInstance;
+    File.getInstance = (() => ({
+      findMany: async (_query: object, options?: { skip?: number; limit?: number }) => {
+        const all = [{ _id: 'file1' }, { _id: 'file2' }];
+        return all.slice(
+          options?.skip ?? 0,
+          (options?.skip ?? 0) + (options?.limit ?? 100),
+        );
+      },
+      deleteMany: async (query: object) => {
+        deletedFiles.push(query);
+      },
+    })) as unknown as typeof File.getInstance;
+
+    const grpcSdk = {
+      authorization: {
+        deleteAllRelations: async (query: { subject?: string; resource?: string }) => {
+          deletedRelations.push(query);
+        },
+      },
+    } as unknown as ConduitGrpcSdk;
+    const storage = {
+      container: () => ({
+        deleteFolder: async () => {
+          providerDeletes += 1;
+          return true;
+        },
+      }),
+    };
+
+    await deleteFolderTree(
+      grpcSdk,
+      storage as never,
+      { _id: 'dir1', name: 'docs/', container: 'conduit' } as never,
+    );
+
+    const resources = deletedRelations.map(item => item.resource).filter(Boolean);
+    const subjects = deletedRelations.map(item => item.subject).filter(Boolean);
+    assert.deepEqual(resources.sort(), [
+      'File:file1',
+      'File:file2',
+      'Folder:dir1',
+      'Folder:dir2',
+    ]);
+    assert.deepEqual(subjects.sort(), ['Folder:dir1', 'Folder:dir2']);
+    assert.equal(providerDeletes, 1);
+    assert.equal(deletedFolders.length, 1);
+    assert.equal(deletedFiles.length, 1);
+    assert.deepEqual(deletedFolders[0], {
+      name: folderPrefixRegex('docs/'),
+      container: 'conduit',
+    });
+  });
+
+  it('succeeds when deleteAllRelations reports no relations found', async () => {
+    ConfigController.getInstance().config = {
+      authorization: { enabled: true },
+      defaultContainer: 'conduit',
+    };
+    _StorageFolder.getInstance = (() => ({
+      findMany: async () => [{ _id: 'dir1' }],
+      deleteMany: async () => undefined,
+    })) as unknown as typeof _StorageFolder.getInstance;
+    File.getInstance = (() => ({
+      findMany: async () => [{ _id: 'file1' }],
+      deleteMany: async () => undefined,
+    })) as unknown as typeof File.getInstance;
+
+    const grpcSdk = {
+      authorization: {
+        deleteAllRelations: async () => {
+          throw new Error('No relations found');
+        },
+      },
+    } as unknown as ConduitGrpcSdk;
+    const storage = {
+      container: () => ({
+        deleteFolder: async () => true,
+      }),
+    };
+
+    await deleteFolderTree(
+      grpcSdk,
+      storage as never,
+      { _id: 'dir1', name: 'docs/', container: 'conduit' } as never,
+    );
+  });
+});
+
+describe('authz disabled cascade', () => {
+  it('does not call deleteAllRelations when authorization is off', async () => {
+    ConfigController.getInstance().config = {
+      authorization: { enabled: false },
+      defaultContainer: 'conduit',
+    };
+    let relationDeletes = 0;
+    _StorageFolder.getInstance = (() => ({
+      findMany: async () => [{ _id: 'dir1' }],
+      deleteMany: async () => undefined,
+    })) as unknown as typeof _StorageFolder.getInstance;
+    File.getInstance = (() => ({
+      findMany: async () => [{ _id: 'file1' }],
+      deleteMany: async () => undefined,
+    })) as unknown as typeof File.getInstance;
+    const grpcSdk = {
+      authorization: {
+        deleteAllRelations: async () => {
+          relationDeletes += 1;
+        },
+      },
+    } as unknown as ConduitGrpcSdk;
+    await deleteFolderTree(
+      grpcSdk,
+      { container: () => ({ deleteFolder: async () => true }) } as never,
+      { _id: 'dir1', name: 'docs/', container: 'conduit' } as never,
+    );
+    assert.equal(relationDeletes, 0);
+  });
+});
+
+describe('deleteContainerTree', () => {
+  it('pages file and folder ids and also clears Container relations', async () => {
+    ConfigController.getInstance().config = {
+      authorization: { enabled: true },
+      defaultContainer: 'conduit',
+    };
+    const deletedRelations: Array<{ subject?: string; resource?: string }> = [];
+    File.getInstance = (() => ({
+      findMany: async (_query: object, options?: { skip?: number; limit?: number }) => {
+        const all = Array.from({ length: 3 }, (_, i) => ({ _id: `file${i}` }));
+        return all.slice(
+          options?.skip ?? 0,
+          (options?.skip ?? 0) + (options?.limit ?? 2),
+        );
+      },
+      deleteMany: async () => undefined,
+    })) as unknown as typeof File.getInstance;
+    _StorageFolder.getInstance = (() => ({
+      findMany: async () => [{ _id: 'dir1' }],
+      deleteMany: async () => undefined,
+    })) as unknown as typeof _StorageFolder.getInstance;
+    _StorageContainer.getInstance = (() => ({
+      deleteOne: async () => undefined,
+    })) as unknown as typeof _StorageContainer.getInstance;
+
+    const grpcSdk = {
+      authorization: {
+        deleteAllRelations: async (query: { subject?: string; resource?: string }) => {
+          deletedRelations.push(query);
+        },
+      },
+    } as unknown as ConduitGrpcSdk;
+    const storage = {
+      deleteContainer: async () => true,
+    };
+
+    await deleteContainerTree(
+      grpcSdk,
+      storage as never,
+      { _id: 'c1', name: 'photos' } as never,
+    );
+
+    assert.equal(
+      deletedRelations.some(item => item.resource === 'Container:c1'),
+      true,
+    );
+    assert.equal(
+      deletedRelations.some(item => item.subject === 'Container:c1'),
+      true,
+    );
+    assert.equal(
+      deletedRelations.filter(item => item.resource?.startsWith('File:')).length,
+      3,
+    );
+  });
+});