From 28f5b1a0617cb4184eea9dd9099cb97dbc7315de Mon Sep 17 00:00:00 2001 From: "Marcelo M. Maciel" <4993482+marcelo-maciel@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:02:29 -0300 Subject: [PATCH 1/3] docs: document Storage:S3:PresignServiceUrl and the compose S3 endpoint --- src/content/docs/building-blocks/storage.mdx | 10 ++++++---- src/content/docs/changelog/index.mdx | 6 +++++- src/content/docs/modules/files.mdx | 4 ++-- 3 files changed, 13 insertions(+), 7 deletions(-) diff --git a/src/content/docs/building-blocks/storage.mdx b/src/content/docs/building-blocks/storage.mdx index ec7c572a..f90351f6 100644 --- a/src/content/docs/building-blocks/storage.mdx +++ b/src/content/docs/building-blocks/storage.mdx @@ -1,6 +1,6 @@ --- title: Storage building block -lastUpdated: 2026-09-25 +lastUpdated: 2026-09-27 description: File storage abstraction - local filesystem or S3-compatible (AWS S3, RustFS, MinIO, R2...) - with presigned URLs, tenant isolation, and optional quota metering. sidebar: label: Storage @@ -56,7 +56,7 @@ public interface IStorageService ### Implementations - **`LocalStorageService`** - stores under `wwwroot/uploads/{owner-type}/{guid}_{sanitized-filename}` and validates extension + size against `FileTypeMetadata.GetRules(fileType)`. Presigning is a **dev-only token fallback**: `GenerateUploadUrlAsync` issues a `local://upload/{token}` URL backed by `LocalPresignTokenStore`; `GenerateDownloadUrlAsync` and `BuildPublicUrl` return server-relative `/uploads/...` paths (no signing needed). -- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer. +- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client for every real S3 call. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer, through a second, keyed `IAmazonS3` (`S3StorageService.PresignClientKey`) aimed at `PresignServiceUrl` when it is set and at `ServiceUrl` otherwise. Presigning is offline, so that client never opens a connection. - **`QuotaMeteredStorageService`** - decorator. `CheckAndRecordAsync(tenantId, QuotaResource.StorageBytes, bytes, ct)` on upload (throws 507 when exceeded, rolls back the charge if the write fails); refunds the object's size on `RemoveAsync`. Requests with no resolved tenant pass through unmetered. ### Request / response @@ -69,7 +69,7 @@ public interface IStorageService ### Options -- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (only applies when `ServiceUrl` is set). +- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `PresignServiceUrl` (optional host that presigned upload/download URLs are signed for, when browsers reach the store on a different address than the API does; `PublicBaseUrl` only rewrites the plain public-read URLs, never a presigned one), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (only applies when `ServiceUrl` is set). ## How modules consume Storage @@ -109,6 +109,7 @@ The generic `T` names the owning type - local storage uses it as the folder segm "S3": { "Bucket": "fsh-uploads", "ServiceUrl": "http://rustfs:9000", // omit for AWS S3 default + "PresignServiceUrl": "https://s3.example.com", // host browsers reach; omit when ServiceUrl is already public "Region": "us-east-1", "ForcePathStyle": true, // required for RustFS / MinIO "AccessKey": "rustfsadmin", @@ -148,10 +149,11 @@ The Files module ships an `IFileScanner` hook. If you're using `IStorageService` ## Gotchas - **Self-hosted S3 stores (RustFS, MinIO) need `ForcePathStyle = true`.** Virtual-hosted-style addressing puts the bucket name in the subdomain, which they can't service without DNS gymnastics. The option defaults to `false` and only takes effect when `ServiceUrl` is set - set it explicitly for RustFS, MinIO, and other self-hosted S3-compatible services. +- **A presigned URL only works on the host it was signed for.** SigV4 signs the `Host` header, so the URL always points at the endpoint of the client that signed it. When the API reaches the store on an internal address a browser cannot resolve (the Docker Compose stack uses `http://rustfs:9000`), set `PresignServiceUrl` to the public one: presigned PUT and GET URLs are signed for it, their scheme follows it (`http` or `https`), and every other S3 call keeps using `ServiceUrl`. The host refuses to start when it is set but is not an absolute `http(s)` URL; left empty, presigning uses `ServiceUrl` exactly as before. A reverse proxy in front of the public endpoint must forward the `Host` header unchanged, or the store answers `403 SignatureDoesNotMatch`, and the store's CORS has to allow the front-end origins, because the browser PUTs to it cross-origin. - **Presigned URLs have a TTL.** Once it expires, the URL is dead. Use `BuildPublicUrl` for non-expiring public URLs (and a bucket policy that grants public-read on that prefix). - **`QuotaMeteredStorageService` is scoped.** It depends on `IQuotaService` which is scoped per request. Don't resolve it from a singleton or a hosted service without creating a scope. - **Local storage paths are not tenant-segmented.** Files land under `wwwroot/uploads/{owner-type}/`, and anything `BuildPublicUrl` points at is served statically without policy enforcement. Don't use `LocalStorageService` in multi-tenant production - it exists for dev and tests. -- **`AddHeroStorage` reads configuration eagerly.** The provider choice and the quota toggle are evaluated once, at registration. Integration tests that swap config after host build must re-register/rewire `IStorageService` post-registration - changing `Storage:Provider` later does nothing. +- **`AddHeroStorage` reads configuration eagerly.** The provider choice and the quota toggle are evaluated once, at registration. Integration tests that swap config after host build must re-register/rewire `IStorageService` post-registration - changing `Storage:Provider` later does nothing. A factory that re-registers the S3 stack this way also has to register the keyed presign client (`S3StorageService.PresignClientKey`, as `FshWebApplicationFactory` does), or `S3StorageService` fails to resolve. ## Critical files diff --git a/src/content/docs/changelog/index.mdx b/src/content/docs/changelog/index.mdx index f01d8a12..049988b4 100644 --- a/src/content/docs/changelog/index.mdx +++ b/src/content/docs/changelog/index.mdx @@ -1,6 +1,6 @@ --- title: Overview -lastUpdated: 2026-09-25 +lastUpdated: 2026-09-27 description: Release notes and version history for fullstackhero. sidebar: order: 1 @@ -11,6 +11,10 @@ seo: Notable changes to the kit, newest first. +## 2026-09-27 + +- **Storage: browser uploads work in the Docker Compose deployment (fix, breaking for docker-compose).** Browsers upload and download through presigned S3 URLs, and SigV4 signs the host, so the API always handed out URLs for the endpoint it talks to itself. In `deploy/docker/docker-compose.yml` that is `http://rustfs:9000`, a compose-internal name a browser cannot resolve, and RustFS published no port anyway, so every file upload from the admin and dashboard failed; `PublicBaseUrl` never applied, because it only rewrites plain public-read URLs. A new optional **`Storage:S3:PresignServiceUrl`** names the host presigned PUT and GET URLs are signed for, and their scheme follows it; every other S3 call keeps using `ServiceUrl`. It is validated at startup as an absolute `http(s)` URL, and left empty the behaviour is unchanged. The compose stack now sets it from a new **`FSH_S3_PUBLIC_URL`**, publishes the RustFS S3 API on `FSH_S3_PORT` (default `9000`), and allows `FSH_ADMIN_URL` and `FSH_DASHBOARD_URL` through `RUSTFS_CORS_ALLOWED_ORIGINS`; `fsh new` fills `FSH_S3_PUBLIC_URL` with `http://localhost:9000`. **Upgrade note:** add `FSH_S3_PUBLIC_URL` to `deploy/docker/.env` (compose refuses to start without it), free host port 9000 or set `FSH_S3_PORT`, and route a TLS hostname at that port with the `Host` header forwarded unchanged, or the store answers `403 SignatureDoesNotMatch`. Aspire and the AWS Terraform stack are unaffected, and on AWS S3 or any store whose own endpoint browsers can reach, leave `PresignServiceUrl` empty. A test factory of your own that re-registers the S3 stack must now also register the keyed presign client (`S3StorageService.PresignClientKey`), as `FshWebApplicationFactory` does. See [Storage](/docs/building-blocks/storage/) and [#PRNUM](https://github.com/fullstackhero/dotnet-starter-kit/pull/PRNUM). + ## 2026-09-25 - **Dependencies: .NET Aspire 13.5.4 and every NuGet package to latest.** The AppHost SDK and `Aspire.Hosting.*` move 13.4.0 → 13.5.4 together (mixing 13.4 and 13.5 packages fails at runtime). The .NET 10 platform packages (ASP.NET Core, EF Core, Extensions, SignalR) go 10.0.8 → 10.0.12, OpenTelemetry 1.15 → 1.19, Asp.Versioning 10.2, Npgsql EF 10.0.3, Scalar 2.17, QuestPDF 2026.9, Hangfire 1.8.25, MailKit/MimeKit 4.18, Testcontainers 4.15, and the rest to latest stable. Three majors: **StackExchange.Redis 3.3** (the same API as 2.13.17 on a rewritten IO core, now **RESP3 by default** - Valkey and ElastiCache both speak it; `Execute("FLUSHALL")`-style admin commands now need `AllowAdmin`), **NSubstitute 6** and **xunit.runner.visualstudio 4** (still runs xUnit v2). `Microsoft.OpenApi` and `MessagePack` stay on their 2.x lines on purpose. The new SonarAnalyzer adds **S8969** (redundant null-forgiving `!`), which is fatal under warnings-as-errors: the kit's own code is cleaned up, but **if you've added code, expect S8969 build errors after pulling** - delete the flagged `!`, and the compiler will tell you if one was actually needed. Asp.Versioning 10.2's `AV0029`/`AV0030` advisories and Aspire's `ASPIRE010` (CLI bundle) are suppressed; the kit keeps one OpenAPI document per version and runs Aspire via `dotnet run`. On the first launch Aspire 13.5 recreates the persistent Postgres and Valkey containers; data volumes are kept and the Postgres image stays on 18, so no wipe is needed. See [#1396](https://github.com/fullstackhero/dotnet-starter-kit/pull/1396). diff --git a/src/content/docs/modules/files.mdx b/src/content/docs/modules/files.mdx index 2e1a3f21..7588696a 100644 --- a/src/content/docs/modules/files.mdx +++ b/src/content/docs/modules/files.mdx @@ -1,6 +1,6 @@ --- title: Files module -lastUpdated: 2026-09-25 +lastUpdated: 2026-09-27 description: Presigned-URL file lifecycle with pluggable per-OwnerType access policies, soft delete + retention purge, optional scanner hook, and S3-compatible storage. sidebar: label: Files @@ -242,7 +242,7 @@ No module ships a consumer today - Catalog and Chat attach files by passing the - `DefaultUploaderOnlyPolicyTests` - policy enforcement - `FileAccessPolicyRegistryTests` - registry lookup - `StorageKeyBuilderTests` - tenant + category path generation -- Integration tests at `src/Tests/Integration.Tests/Tests/Files/` (ten files) cover the presigned round-trip (`RequestAndFinalizeUploadTests`, `StorageFlowTests`), upload validation, finalize edge cases, visibility + sharing, soft delete + restore, the purge jobs, and tenant isolation against Testcontainers RustFS. +- Integration tests at `src/Tests/Integration.Tests/Tests/Files/` (eleven files) cover the presigned round-trip (`RequestAndFinalizeUploadTests`, `StorageFlowTests`), presigned URLs signed for a separate public endpoint (`PresignServiceUrlTests`), upload validation, finalize edge cases, visibility + sharing, soft delete + restore, the purge jobs, and tenant isolation against Testcontainers RustFS. ## Related From ea934c47d3b4ce3740ad2c1b316dcf3ebee25604 Mon Sep 17 00:00:00 2001 From: "Marcelo M. Maciel" <4993482+marcelo-maciel@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:11:58 -0300 Subject: [PATCH 2/3] docs(storage): correct what the presign client does and document the no-path rule --- src/content/docs/building-blocks/storage.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/docs/building-blocks/storage.mdx b/src/content/docs/building-blocks/storage.mdx index f90351f6..7ee7bce7 100644 --- a/src/content/docs/building-blocks/storage.mdx +++ b/src/content/docs/building-blocks/storage.mdx @@ -56,7 +56,7 @@ public interface IStorageService ### Implementations - **`LocalStorageService`** - stores under `wwwroot/uploads/{owner-type}/{guid}_{sanitized-filename}` and validates extension + size against `FileTypeMetadata.GetRules(fileType)`. Presigning is a **dev-only token fallback**: `GenerateUploadUrlAsync` issues a `local://upload/{token}` URL backed by `LocalPresignTokenStore`; `GenerateDownloadUrlAsync` and `BuildPublicUrl` return server-relative `/uploads/...` paths (no signing needed). -- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client for every real S3 call. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer, through a second, keyed `IAmazonS3` (`S3StorageService.PresignClientKey`) aimed at `PresignServiceUrl` when it is set and at `ServiceUrl` otherwise. Presigning is offline, so that client never opens a connection. +- **`S3StorageService`** - uses `AWSSDK.S3` with a singleton `IAmazonS3` client for every real S3 call. A custom `ServiceUrl` (RustFS, MinIO, etc.) switches to path-style addressing per `ForcePathStyle`; presigned PUT/GET URLs come from the SDK's request signer, through a second, keyed `IAmazonS3` (`S3StorageService.PresignClientKey`) aimed at `PresignServiceUrl` when it is set and at `ServiceUrl` otherwise. Signing itself is offline; the only network call that client can make is fetching ambient credentials when no keys are set. - **`QuotaMeteredStorageService`** - decorator. `CheckAndRecordAsync(tenantId, QuotaResource.StorageBytes, bytes, ct)` on upload (throws 507 when exceeded, rolls back the charge if the write fails); refunds the object's size on `RemoveAsync`. Requests with no resolved tenant pass through unmetered. ### Request / response @@ -69,7 +69,7 @@ public interface IStorageService ### Options -- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `PresignServiceUrl` (optional host that presigned upload/download URLs are signed for, when browsers reach the store on a different address than the API does; `PublicBaseUrl` only rewrites the plain public-read URLs, never a presigned one), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (only applies when `ServiceUrl` is set). +- **`S3StorageOptions`** - `Bucket`, `Region`, `Prefix`, `PublicRead` (default true), `PublicBaseUrl` (for non-expiring public URLs), `ServiceUrl` (custom endpoint for RustFS, MinIO, etc.), `PresignServiceUrl` (optional host that presigned upload/download URLs are signed for, when browsers reach the store on a different address than the API does; `PublicBaseUrl` only rewrites the plain public-read URLs, never a presigned one), `AccessKey` / `SecretKey` (leave empty to use the AWS SDK credential chain), `ForcePathStyle` (applies to each client with a custom endpoint, `ServiceUrl` or `PresignServiceUrl`). `PresignServiceUrl` must be an absolute http(s) URL with no path, checked at startup: a path would be signed into every URL and break behind a proxy. ## How modules consume Storage From ae84111b1922611f07a5c7f477f01e6f0484f915 Mon Sep 17 00:00:00 2001 From: "Marcelo M. Maciel" <4993482+marcelo-maciel@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:32:16 -0300 Subject: [PATCH 3/3] docs(changelog): link the PresignServiceUrl entry to #1402 --- src/content/docs/changelog/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/changelog/index.mdx b/src/content/docs/changelog/index.mdx index 049988b4..8ed46767 100644 --- a/src/content/docs/changelog/index.mdx +++ b/src/content/docs/changelog/index.mdx @@ -13,7 +13,7 @@ Notable changes to the kit, newest first. ## 2026-09-27 -- **Storage: browser uploads work in the Docker Compose deployment (fix, breaking for docker-compose).** Browsers upload and download through presigned S3 URLs, and SigV4 signs the host, so the API always handed out URLs for the endpoint it talks to itself. In `deploy/docker/docker-compose.yml` that is `http://rustfs:9000`, a compose-internal name a browser cannot resolve, and RustFS published no port anyway, so every file upload from the admin and dashboard failed; `PublicBaseUrl` never applied, because it only rewrites plain public-read URLs. A new optional **`Storage:S3:PresignServiceUrl`** names the host presigned PUT and GET URLs are signed for, and their scheme follows it; every other S3 call keeps using `ServiceUrl`. It is validated at startup as an absolute `http(s)` URL, and left empty the behaviour is unchanged. The compose stack now sets it from a new **`FSH_S3_PUBLIC_URL`**, publishes the RustFS S3 API on `FSH_S3_PORT` (default `9000`), and allows `FSH_ADMIN_URL` and `FSH_DASHBOARD_URL` through `RUSTFS_CORS_ALLOWED_ORIGINS`; `fsh new` fills `FSH_S3_PUBLIC_URL` with `http://localhost:9000`. **Upgrade note:** add `FSH_S3_PUBLIC_URL` to `deploy/docker/.env` (compose refuses to start without it), free host port 9000 or set `FSH_S3_PORT`, and route a TLS hostname at that port with the `Host` header forwarded unchanged, or the store answers `403 SignatureDoesNotMatch`. Aspire and the AWS Terraform stack are unaffected, and on AWS S3 or any store whose own endpoint browsers can reach, leave `PresignServiceUrl` empty. A test factory of your own that re-registers the S3 stack must now also register the keyed presign client (`S3StorageService.PresignClientKey`), as `FshWebApplicationFactory` does. See [Storage](/docs/building-blocks/storage/) and [#PRNUM](https://github.com/fullstackhero/dotnet-starter-kit/pull/PRNUM). +- **Storage: browser uploads work in the Docker Compose deployment (fix, breaking for docker-compose).** Browsers upload and download through presigned S3 URLs, and SigV4 signs the host, so the API always handed out URLs for the endpoint it talks to itself. In `deploy/docker/docker-compose.yml` that is `http://rustfs:9000`, a compose-internal name a browser cannot resolve, and RustFS published no port anyway, so every file upload from the admin and dashboard failed; `PublicBaseUrl` never applied, because it only rewrites plain public-read URLs. A new optional **`Storage:S3:PresignServiceUrl`** names the host presigned PUT and GET URLs are signed for, and their scheme follows it; every other S3 call keeps using `ServiceUrl`. It is validated at startup as an absolute `http(s)` URL with no path, and left empty the behaviour is unchanged. The compose stack now sets it from a new **`FSH_S3_PUBLIC_URL`**, publishes the RustFS S3 API on `FSH_S3_PORT` (default `9000`), and allows `FSH_ADMIN_URL` and `FSH_DASHBOARD_URL` through `RUSTFS_CORS_ALLOWED_ORIGINS`; `fsh new` fills `FSH_S3_PUBLIC_URL` with `http://localhost:9000`. **Upgrade note:** add `FSH_S3_PUBLIC_URL` to `deploy/docker/.env` (compose refuses to start without it), free host port 9000 or set `FSH_S3_PORT`, and route a TLS hostname at that port with the `Host` header forwarded unchanged, or the store answers `403 SignatureDoesNotMatch`. Aspire and the AWS Terraform stack are unaffected, and on AWS S3 or any store whose own endpoint browsers can reach, leave `PresignServiceUrl` empty. A test factory of your own that re-registers the S3 stack must now also register the keyed presign client (`S3StorageService.PresignClientKey`), as `FshWebApplicationFactory` does. See [Storage](/docs/building-blocks/storage/) and [#1402](https://github.com/fullstackhero/dotnet-starter-kit/pull/1402). ## 2026-09-25