diff --git a/src/content/docs/building-blocks/storage.mdx b/src/content/docs/building-blocks/storage.mdx index ec7c572a..7ee7bce7 100644 --- a/src/content/docs/building-blocks/storage.mdx +++ b/src/content/docs/building-blocks/storage.mdx @@ -1,6 +1,6 @@ --- title: Storage building block -lastUpdated: 2026-09-25 +lastUpdated: 2026-09-27 description: File storage abstraction - local filesystem or S3-compatible (AWS S3, RustFS, MinIO, R2...) - with presigned URLs, tenant isolation, and optional quota metering. sidebar: label: Storage @@ -56,7 +56,7 @@ public interface IStorageService ### Implementations - **`LocalStorageService`** - stores under `wwwroot/uploads/{owner-type}/{guid}_{sanitized-filename}` and validates extension + size against `FileTypeMetadata.GetRules(fileType)`. Presigning is a **dev-only token fallback**: `GenerateUploadUrlAsync` issues a `local://upload/{token}` URL backed by `LocalPresignTokenStore`; `GenerateDownloadUrlAsync` and `BuildPublicUrl` return server-relative `/uploads/...` paths (no signing needed). -- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer. +- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client for every real S3 call. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer, through a second, keyed `IAmazonS3` (`S3StorageService.PresignClientKey`) aimed at `PresignServiceUrl` when it is set and at `ServiceUrl` otherwise. Signing itself is offline; the only network call that client can make is fetching ambient credentials when no keys are set. - **`QuotaMeteredStorageService`** - decorator. `CheckAndRecordAsync(tenantId, QuotaResource.StorageBytes, bytes, ct)` on upload (throws 507 when exceeded, rolls back the charge if the write fails); refunds the object's size on `RemoveAsync`. Requests with no resolved tenant pass through unmetered. ### Request / response @@ -69,7 +69,7 @@ public interface IStorageService ### Options -- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (only applies when `ServiceUrl` is set). +- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `PresignServiceUrl` (optional host that presigned upload/download URLs are signed for, when browsers reach the store on a different address than the API does; `PublicBaseUrl` only rewrites the plain public-read URLs, never a presigned one), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (applies to each client with a custom endpoint, `ServiceUrl` or `PresignServiceUrl`). `PresignServiceUrl` must be an absolute http(s) URL with no path, checked at startup: a path would be signed into every URL and break behind a proxy. ## How modules consume Storage @@ -109,6 +109,7 @@ The generic `T` names the owning type - local storage uses it as the folder segm "S3": { "Bucket": "fsh-uploads", "ServiceUrl": "http://rustfs:9000", // omit for AWS S3 default + "PresignServiceUrl": "https://s3.example.com", // host browsers reach; omit when ServiceUrl is already public "Region": "us-east-1", "ForcePathStyle": true, // required for RustFS / MinIO "AccessKey": "rustfsadmin", @@ -148,10 +149,11 @@ The Files module ships an `IFileScanner` hook. If you're using `IStorageService` ## Gotchas - **Self-hosted S3 stores (RustFS, MinIO) need `ForcePathStyle = true`.** Virtual-hosted-style addressing puts the bucket name in the subdomain, which they can't service without DNS gymnastics. The option defaults to `false` and only takes effect when `ServiceUrl` is set - set it explicitly for RustFS, MinIO, and other self-hosted S3-compatible services. +- **A presigned URL only works on the host it was signed for.** SigV4 signs the `Host` header, so the URL always points at the endpoint of the client that signed it. When the API reaches the store on an internal address a browser cannot resolve (the Docker Compose stack uses `http://rustfs:9000`), set `PresignServiceUrl` to the public one: presigned PUT and GET URLs are signed for it, their scheme follows it (`http` or `https`), and every other S3 call keeps using `ServiceUrl`. The host refuses to start when it is set but is not an absolute `http(s)` URL; left empty, presigning uses `ServiceUrl` exactly as before. A reverse proxy in front of the public endpoint must forward the `Host` header unchanged, or the store answers `403 SignatureDoesNotMatch`, and the store's CORS has to allow the front-end origins, because the browser PUTs to it cross-origin. - **Presigned URLs have a TTL.** Once it expires, the URL is dead. Use `BuildPublicUrl` for non-expiring public URLs (and a bucket policy that grants public-read on that prefix). - **`QuotaMeteredStorageService` is scoped.** It depends on `IQuotaService` which is scoped per request. Don't resolve it from a singleton or a hosted service without creating a scope. - **Local storage paths are not tenant-segmented.** Files land under `wwwroot/uploads/{owner-type}/`, and anything `BuildPublicUrl` points at is served statically without policy enforcement. Don't use `LocalStorageService` in multi-tenant production - it exists for dev and tests. -- **`AddHeroStorage` reads configuration eagerly.** The provider choice and the quota toggle are evaluated once, at registration. Integration tests that swap config after host build must re-register/rewire `IStorageService` post-registration - changing `Storage:Provider` later does nothing. +- **`AddHeroStorage` reads configuration eagerly.** The provider choice and the quota toggle are evaluated once, at registration. Integration tests that swap config after host build must re-register/rewire `IStorageService` post-registration - changing `Storage:Provider` later does nothing. A factory that re-registers the S3 stack this way also has to register the keyed presign client (`S3StorageService.PresignClientKey`, as `FshWebApplicationFactory` does), or `S3StorageService` fails to resolve. ## Critical files diff --git a/src/content/docs/changelog/index.mdx b/src/content/docs/changelog/index.mdx index cf5b6c84..ea25127e 100644 --- a/src/content/docs/changelog/index.mdx +++ b/src/content/docs/changelog/index.mdx @@ -1,6 +1,6 @@ --- title: Overview -lastUpdated: 2026-09-26 +lastUpdated: 2026-09-27 description: Release notes and version history for fullstackhero. sidebar: order: 1 @@ -11,6 +11,10 @@ seo: Notable changes to the kit, newest first. +## 2026-09-27 + +- **Storage: browser uploads work in the Docker Compose deployment (fix, breaking for docker-compose).** Browsers upload and download through presigned S3 URLs, and SigV4 signs the host, so the API always handed out URLs for the endpoint it talks to itself. In `deploy/docker/docker-compose.yml` that is `http://rustfs:9000`, a compose-internal name a browser cannot resolve, and RustFS published no port anyway, so every file upload from the admin and dashboard failed; `PublicBaseUrl` never applied, because it only rewrites plain public-read URLs. A new optional **`Storage:S3:PresignServiceUrl`** names the host presigned PUT and GET URLs are signed for, and their scheme follows it; every other S3 call keeps using `ServiceUrl`. It is validated at startup as an absolute `http(s)` URL with no path, and left empty the behaviour is unchanged. The compose stack now sets it from a new **`FSH_S3_PUBLIC_URL`**, publishes the RustFS S3 API on `FSH_S3_PORT` (default `9000`), and allows `FSH_ADMIN_URL` and `FSH_DASHBOARD_URL` through `RUSTFS_CORS_ALLOWED_ORIGINS`; `fsh new` fills `FSH_S3_PUBLIC_URL` with `http://localhost:9000`. **Upgrade note:** add `FSH_S3_PUBLIC_URL` to `deploy/docker/.env` (compose refuses to start without it), free host port 9000 or set `FSH_S3_PORT`, and route a TLS hostname at that port with the `Host` header forwarded unchanged, or the store answers `403 SignatureDoesNotMatch`. Aspire and the AWS Terraform stack are unaffected, and on AWS S3 or any store whose own endpoint browsers can reach, leave `PresignServiceUrl` empty. A test factory of your own that re-registers the S3 stack must now also register the keyed presign client (`S3StorageService.PresignClientKey`), as `FshWebApplicationFactory` does. See [Storage](/docs/building-blocks/storage/) and [#1402](https://github.com/fullstackhero/dotnet-starter-kit/pull/1402). + ## 2026-09-26 - **Frontend containers: admin and dashboard survive a restart (fix).** Both images rendered `/config.json` from `/usr/share/nginx/html/config.json.template` at startup and then deleted the template so it wouldn't be served. A container's filesystem outlives a restart, so the next start of the same container - `docker restart`, a Docker daemon restart, or a host reboot under `restart: unless-stopped` - found no template, failed with `can't open /usr/share/nginx/html/config.json.template: no such file` under `set -e`, and crash-looped forever. The template now lives at `/etc/fsh/config.json.template`, outside the web root, so it is still never served, and `config.json` is re-rendered from it on every start. The files copied into both images are now checked out with LF line endings on every platform, so an image built on Windows no longer serves `config.json` with CRLF. **Upgrade note:** rebuild the admin and dashboard images. A container already stuck in the loop is fixed by recreating it (`docker compose up -d --force-recreate admin dashboard`), even before you upgrade - a fresh container gets the template back until its next restart. See [#1400](https://github.com/fullstackhero/dotnet-starter-kit/pull/1400). diff --git a/src/content/docs/modules/files.mdx b/src/content/docs/modules/files.mdx index 2e1a3f21..7588696a 100644 --- a/src/content/docs/modules/files.mdx +++ b/src/content/docs/modules/files.mdx @@ -1,6 +1,6 @@ --- title: Files module -lastUpdated: 2026-09-25 +lastUpdated: 2026-09-27 description: Presigned-URL file lifecycle with pluggable per-OwnerType access policies, soft delete + retention purge, optional scanner hook, and S3-compatible storage. sidebar: label: Files @@ -242,7 +242,7 @@ No module ships a consumer today - Catalog and Chat attach files by passing the - `DefaultUploaderOnlyPolicyTests` - policy enforcement - `FileAccessPolicyRegistryTests` - registry lookup - `StorageKeyBuilderTests` - tenant + category path generation -- Integration tests at `src/Tests/Integration.Tests/Tests/Files/` (ten files) cover the presigned round-trip (`RequestAndFinalizeUploadTests`, `StorageFlowTests`), upload validation, finalize edge cases, visibility + sharing, soft delete + restore, the purge jobs, and tenant isolation against Testcontainers RustFS. +- Integration tests at `src/Tests/Integration.Tests/Tests/Files/` (eleven files) cover the presigned round-trip (`RequestAndFinalizeUploadTests`, `StorageFlowTests`), presigned URLs signed for a separate public endpoint (`PresignServiceUrlTests`), upload validation, finalize edge cases, visibility + sharing, soft delete + restore, the purge jobs, and tenant isolation against Testcontainers RustFS. ## Related