diff --git a/documentation/docs/README.md b/documentation/docs/README.md index 2489f1f..3aba545 100644 --- a/documentation/docs/README.md +++ b/documentation/docs/README.md @@ -121,7 +121,7 @@ Your server is now running at `http://localhost:8000`. | `vania_elasticsearch` | Elasticsearch client with query builder | 1.0.0 | | `vania_graphql` | GraphQL server with subscriptions | 1.0.0 | | `vania_grpc` | gRPC server and client | 1.0.0 | -| `vania_swagger` | OpenAPI 3.0 documentation and Swagger UI | 1.0.0 | +| `vania_swagger` | OpenAPI 3.2 documentation and Swagger UI | 1.0.0 | | `vania_websocket` | WebSocket channels, presence, and broadcasting | 1.0.0 | | `vania_cli` | CLI for scaffolding, serving, and migrations | 1.0.0 | diff --git a/documentation/docs/core/error-handling.md b/documentation/docs/core/error-handling.md index 66c0e97..710c005 100644 --- a/documentation/docs/core/error-handling.md +++ b/documentation/docs/core/error-handling.md @@ -24,7 +24,7 @@ throw ForbiddenException(message: 'You do not have access'); // 401 Unauthorized throw Unauthenticated(message: 'Invalid credentials'); -// 500 Internal Server Error +// 500 Internal Server Error — the client sees the message only in debug throw InternalServerError(message: 'Something went wrong'); // Custom HTTP status — the class is HttpResponseException, with a `code` @@ -66,6 +66,8 @@ class AdminMiddleware extends Middleware { See [Middleware](middleware.md) for the full `handle` / `process` model. +An exception gets the same response whether a middleware or the controller threw it. + ## Error Response Format ### JSON Requests @@ -78,15 +80,17 @@ For API requests, exceptions are returned as JSON: } ``` +A 5xx exception you throw yourself is treated like any other server failure: it is logged, and its message reaches the client only when `APP_DEBUG=true`. Messages of 4xx exceptions are written for the client and always shown. + ### HTML Requests -For web requests, the framework renders error view templates if they exist: +A request that asks for HTML, in an app that renders pages (it has views, or CSRF protection on), gets HTML instead: -- `views/errors/404.html` for 404 errors -- `views/errors/500.html` for 500 errors -- `views/errors/419.html` for CSRF failures +- `RedirectException` redirects to its `message`. So does an `Unauthenticated` created with `responseType: ResponseType.html`, which is how a session guard sends a browser to its login page: `Unauthenticated(message: '/login', responseType: ResponseType.html)`. Any other `Unauthenticated` is a `401`, even for a browser — its message is text for the client, not an address. +- A 404, 500 or 419 (CSRF failure) renders `lib/resources/view/errors/404.html`, `500.html` or `419.html`, when it exists. +- Anything else, or a missing view, is a plain HTML page with the status code and the escaped message. -If the view file does not exist, a plain text response is returned. +An app without views answers in JSON, even a browser. ## Validation Errors @@ -102,7 +106,7 @@ Validation failures return a `422` response with structured error data: } ``` -For HTML requests, the user is redirected back with errors and old input in the session. +For HTML requests, the user is redirected back with errors and old input in the session. An app without sessions (no views and CSRF off) has nowhere to keep them across a redirect, so it answers HTML requests with the `422` too. ## Database Errors @@ -110,4 +114,4 @@ For HTML requests, the user is redirected back with errors and old input in the ## Unhandled Exceptions -Any exception not caught by your code is caught by the request pipeline. It logs the error and returns a 500 response. In debug mode, the exception message and stack trace are included in the response. +Any exception not caught by your code is caught by the request pipeline. It logs the error with its stack trace and returns a 500 response. In debug mode, the exception message is included in the response; the stack trace only goes to the log. diff --git a/documentation/docs/core/middleware.md b/documentation/docs/core/middleware.md index 9adc8f6..c4cf631 100644 --- a/documentation/docs/core/middleware.md +++ b/documentation/docs/core/middleware.md @@ -47,7 +47,39 @@ class Timing extends Middleware { } ``` -Not calling `next()` stops the chain: no later middleware runs, the controller is never invoked, and your middleware becomes responsible for the response. Override **either** `handle` or `process`, not both — if you define `process`, `handle` is ignored. +The response is written only once every middleware has finished, so the header above still reaches the client. An exception from the controller or a later middleware comes back out of `await next()`: a `process` middleware can catch it to log or measure it, and should rethrow it so it is still answered. + +Not calling `next()` stops the chain: no later middleware runs, the controller is never invoked, and your middleware becomes responsible for the response. Calling it twice is an error, since it would run the controller twice. Override **either** `handle` or `process`, not both — if you define `process`, `handle` is ignored. + +## Reading the body in middleware + +Middleware runs **before** the request body is read, so `req.input(...)` in a middleware sees only the query string and route parameters. This is deliberate: CSRF protection checks the token before it lets the framework read a possibly large body. (With CSRF protection on, a `POST`, `PUT`, `PATCH` or `DELETE` that passed the check has had its body read by it — but don't rely on that.) A middleware that needs the body reads it first: + +```dart +@override +Future handle(Request req) async { + await req.extractBody(); + final email = req.input('email'); + // ... +} +``` + +Reading it here costs nothing extra later: the body is read once, and the controller gets the same input. + +## One instance for every request + +The middleware objects you attach to routes are created once, when routes are registered, and shared by every request — including requests running at the same time. Keep per-request data out of their fields: + +```dart +class Bad extends Middleware { + String? userId; // another request can overwrite this before you read it + + @override + Future handle(Request req) async => userId = req.header('x-user'); +} +``` + +Pass such data on with `req.merge({...})` instead, as in the first example. Fields that hold configuration, or state meant to be shared such as `Throttle`'s counters, are fine. ## Attaching middleware to routes @@ -75,10 +107,12 @@ Group middleware runs before route middleware, and each list runs in order. import 'package:vania/http/middleware.dart'; Router.post('/login', authController.login) - .middleware([Throttle(maxRequests: 5, perMinutes: 1)]); + .middleware([Throttle(maxAttempts: 5, duration: Duration(minutes: 1))]); ``` -Five requests per minute per client; excess requests get `429 Too Many Requests`. +Five requests per minute per client; excess requests get `429 Too Many Requests` with a `Retry-After` header. Every response carries `X-RateLimit-Limit`, `X-RateLimit-Remaining` (what is left after this request) and `X-RateLimit-Reset` (when the count starts over). + +Clients are told apart by IP address, which behind a reverse proxy is the proxy's unless it is listed in `TRUSTED_PROXIES`. Pass `includeUserIdentifier: true` to count each signed-in user separately. Pass `bypassInDevelopment: true` to skip throttling while `APP_ENV` is `local` or `development`; it is off by default, so a production app that kept a development `.env` is still rate limited. The counts are kept in memory, so they are per process, and per isolate with `APP_ISOLATES`. ### Security headers @@ -122,9 +156,10 @@ Router.websocket('/ws', eventHandler, middleware: [WsAuth()]); ## Execution order -1. Group middleware (outer groups first, then inner) -2. Route middleware, in the order listed -3. The controller action +1. CSRF protection, when it is on and the request changes state (see [Sessions](sessions.md#csrf-tokens)) +2. Group middleware (outer groups first, then inner) +3. Route middleware, in the order listed +4. The controller action A middleware that throws, or that overrides `process` and doesn't call `next()`, stops the chain there. diff --git a/documentation/docs/core/requests.md b/documentation/docs/core/requests.md index 1d311bd..ace1d36 100644 --- a/documentation/docs/core/requests.md +++ b/documentation/docs/core/requests.md @@ -192,6 +192,16 @@ Parse a specific field as JSON: Map metadata = req.json('metadata'); ``` +## Raw Content + +`req.rawBody` is the request content exactly as received, as bytes. Use it for a media type the framework does not read into fields (the SQL or GraphQL of a [`QUERY`](routing.md#query-routes) request, say) or to verify a webhook signature over the bytes that were signed: + +```dart +final text = utf8.decode(req.rawBody); +``` + +It is empty for multipart uploads, which are streamed rather than held in memory. + ## Authenticated User If the authentication package is installed and the route uses auth middleware: diff --git a/documentation/docs/core/responses.md b/documentation/docs/core/responses.md index db7e176..1526e02 100644 --- a/documentation/docs/core/responses.md +++ b/documentation/docs/core/responses.md @@ -76,6 +76,8 @@ return Response.back('status', 'Profile updated'); return Response.backWithInput('email', 'That address is already taken'); ``` +The previous page is taken from the `Referer` header, and only when it points at this same site, so a link from elsewhere cannot use `back()` to redirect off your site. Without one, `back()` from a form post returns to the page at the same path, and from anything else goes to `/` — never to the URL it was called from, which would redirect to itself. + ## File Responses ### Stream a File @@ -101,6 +103,25 @@ return Response.download( ); ``` +### Serve a File from Disk + +`Response.file` and `Response.download` send bytes you already hold in memory. For a file on disk — a video, a large export — use `Response.fromPath`, which streams it without loading it into memory: + +```dart +return Response.fromPath('storage/videos/intro.mp4'); + +// Saved by the browser, under a name of your choosing +return Response.fromPath( + 'storage/exports/2026-10.csv', + download: true, + fileName: 'october.csv', +); +``` + +It answers `Range` requests with `206 Partial Content`, so video players can seek and interrupted downloads can resume. A path that is not a file gets a 404. + +The path is read as given. Never build it from request input without confining it to a directory you mean to serve. + ## Server-Sent Events (SSE) Stream real-time events to the client: @@ -122,6 +143,8 @@ Future notifications(Request req) async { } ``` +Each event is sent as soon as the stream yields it. While the stream is quiet, a comment line goes out every `SSE_HEARTBEAT` seconds (default 15), which keeps proxies from closing the connection. When the client goes away, the subscription to your stream is cancelled, so a `StreamController`'s `onCancel` is the place to release whatever feeds it. An error from the stream ends the response and is logged. + ## Empty / No Content Return an empty 204 response: diff --git a/documentation/docs/core/routing.md b/documentation/docs/core/routing.md index 51d4b46..a3a98b3 100644 --- a/documentation/docs/core/routing.md +++ b/documentation/docs/core/routing.md @@ -42,11 +42,39 @@ Router.put(path, action); Router.patch(path, action); Router.delete(path, action); Router.options(path, action); +Router.query(path, action); // HTTP QUERY, see below Router.any(path, action); // matches all methods ``` Additional methods for specialized protocols: `Router.purge`, `Router.copy`, `Router.link`, `Router.unlink`, `Router.lock`, `Router.unlock`, `Router.propfind`. +### QUERY Routes + +`QUERY` ([RFC 10008](https://www.rfc-editor.org/rfc/rfc10008)) is a read like `GET`, safe and idempotent, that carries the query in the request content as `POST` does. Use it when a query is too large or too complex for a URL, or too sensitive to leave in one: URLs end up in logs and browser history, request content generally does not. + +```dart +Router.query('/contacts', contactController.search) + .acceptQuery(['application/json', 'application/x-www-form-urlencoded']); +``` + +A JSON or form-encoded query reaches the action as ordinary input, so `req.input('surname')` and `req.validate(...)` work as they do for a `POST`. Any other media type, such as SQL or GraphQL, is in `req.rawBody`: + +```dart +Router.query('/reports', (Request req) { + final sql = utf8.decode(req.rawBody); // Content-Type: application/sql + // ... +}); +``` + +The framework applies the RFC's rules for you: + +- A `QUERY` without a `Content-Type`, or whose content does not parse as that type, is answered with **400**. +- With `acceptQuery`, a request of any other media type gets **415**, and responses carry an `Accept-Query` header listing the accepted types. `type/*` and `*/*` are allowed. +- An `OPTIONS` request to the path answers with an `Allow` header naming its methods (`GET, QUERY, OPTIONS`) and, when set, `Accept-Query`. That is how a client finds out a resource supports `QUERY`. +- `QUERY` is safe, so CSRF protection does not apply to it, and `Response.fromPath` honours `Range` for it as for `GET`. + +Browsers cannot send `QUERY` from an HTML form, only with `fetch()`. A cross-origin `QUERY` is preflighted, so add `QUERY` to the CORS `methods` (see [Configuration](../getting-started/configuration.md#cors)). + ## Route Actions A route action can be a controller method or an inline closure: @@ -73,11 +101,11 @@ Router.get('/users/{id}', userController.show); Router.get('/posts/{postId}/comments/{commentId}', commentController.show); ``` -Parameters are passed as positional arguments to the action. The framework auto-parses `int`, `double`, and `bool` values based on the action's type signature. +Parameters are passed as positional arguments to the action, decoded: a request for `/posts/%D8%B3%D9%84%D8%A7%D9%85` gives `سلام`. Routes may be written in any script — `Router.get('/مقالات/{id}', ...)` matches the encoded request a browser sends for it. An unconstrained parameter is converted from its text: `42` arrives as an `int`, `4.2` as a `double`, `true` or `false` as a `bool`, and anything else as a `String`. To receive digits as a `String` (a slug such as `2024`, say), constrain it with `whereString`. ### Parameter Type Constraints -Constrain parameter types to reject non-matching requests early: +Constrain parameter types to reject non-matching requests early. A constrained parameter reaches the action as that type, and a value that does not fit is a 404: ```dart Router.get('/users/{id}', userController.show) @@ -89,11 +117,16 @@ Router.get('/price/{amount}', handler) Router.get('/active/{flag}', handler) .whereBool('flag'); +Router.get('/archive/{year}', handler) + .whereString('year'); // "2024" stays a String + // Custom regex constraint Router.get('/slug/{slug}', handler) .where('slug', r'[a-z0-9-]+'); ``` +A `where` pattern has to match the whole segment: `r'\d+'` takes `123` but not `abc123`. + ## Route Groups Group related routes to share a prefix, middleware, or domain: diff --git a/documentation/docs/core/sessions.md b/documentation/docs/core/sessions.md index b732a82..3e63a6b 100644 --- a/documentation/docs/core/sessions.md +++ b/documentation/docs/core/sessions.md @@ -53,9 +53,28 @@ Access the current session identifier: String? sessionId = SessionManager().getSessionId(); ``` +### After signing in + +Give the session a new id whenever a user signs in, or their privileges change: + +```dart +await regenerateSession(); +await setSession('logged_in', true); +``` + +The data stays; only the id changes, and the old one stops working. Someone who learned the old id — by planting a cookie in the browser before the user signed in, say — is left holding nothing. For the same reason, an id the server never issued is not taken up: a request carrying one starts a fresh session with an id of the server's choosing. That holds for an API call made with `fetch` as much as for a page, so a session written by a JSON login lands under a new id too. + ## CSRF Tokens -CSRF protection for web forms is handled for you. In templates, emit the token with the `@csrf` directive (a hidden form field) or `@csrfToken` (the raw value) — see [Views & Templates](views-templates.md). The session layer generates and validates the token behind the scenes; you do not read it from `SessionManager` directly. +CSRF protection is off until you turn it on: + +```env +CSRF_PROTECTION_ENABLED=true +``` + +From then on every `POST`, `PUT`, `PATCH` and `DELETE` request must carry the session's token, or it is refused with a `419`. `GET`, `HEAD`, `OPTIONS` and `QUERY` change nothing, so they need none. Routes under `api/` are left out, since an API normally authenticates with a token rather than the session cookie; set `csrf_protect_api: true` in the config map if yours uses the cookie, and list any other paths to leave out under `csrf_except`. + +In templates, emit the token with the `@csrf` directive (a hidden form field) or `@csrfToken` (the raw value) — see [Views & Templates](views-templates.md). A request sent from JavaScript, which is the only way to send a `PUT`, `PATCH` or `DELETE` from a page, passes it in the `X-CSRF-TOKEN` header instead. The session layer generates and validates the token behind the scenes; you do not read it from `SessionManager` directly. ## Session Configuration @@ -63,9 +82,11 @@ Sessions are configured through environment variables: | Variable | Default | Description | |----------|---------|-------------| -| `SESSION_LIFETIME` | `9000` | Session duration in seconds | +| `SESSION_LIFETIME` | `9000` | Seconds a session lasts, for both the cookie and the stored session | | `SESSION_CACHE_SIZE` | `5000` | Maximum sessions cached in memory | -| `SECURE_SESSION` | `false` | Set `true` to enable the `Secure` cookie flag | +| `SECURE_SESSION` | `true` | Sends the cookie only over HTTPS; set `false` for local development over plain HTTP | + +A session that is in use stays alive: once it is past half its lifetime, the next request renews it. So an idle session ends between half and all of `SESSION_LIFETIME` after its last request. Expired session files are swept from disk about once an hour. Session files are stored in `storage/framework/sessions/`. Each session is AES-GCM encrypted using `APP_KEY`, and session IDs are HMAC-SHA256 hashed for filenames. @@ -73,11 +94,11 @@ The session cookie is named `{APP_NAME}_session`, is `httpOnly`, and uses `SameS ## How Sessions Work -1. When an HTML request arrives, the `SessionManager` checks for the session cookie. -2. If a session ID exists, it loads the encrypted session data from the file store (or in-memory LRU cache). -3. If no session exists, a new one is created. -4. At the end of the request, session data is persisted to the file store. -5. API requests (JSON content type) skip session handling entirely for performance. +1. When a request asks for HTML (its `Accept` header) in an app that renders pages, the `SessionManager` checks for the session cookie. +2. If the cookie names an existing session, its encrypted data is loaded from the file store (or the in-memory LRU cache). +3. Otherwise a new session is started, under a new id. +4. Changes are saved once, just before the response is written, and only if there were any. +5. Every other request — an API call, or any request to an app without views — has no session at all. ## Using Sessions in Templates diff --git a/documentation/docs/core/validation.md b/documentation/docs/core/validation.md index 07eff70..9b31ef4 100644 --- a/documentation/docs/core/validation.md +++ b/documentation/docs/core/validation.md @@ -210,4 +210,4 @@ When validation fails, the framework returns a `422` response with this structur } ``` -For HTML requests (web routes), the user is redirected back with errors and old input available in the session. +For HTML requests (web routes), the user is redirected back with errors and old input available in the session. They are kept for the next request only. Fields whose name contains `password` are never kept as old input, and neither are uploaded files. An app without sessions (no views and CSRF off) has nowhere to keep them across a redirect, so it answers HTML requests with the `422` too. diff --git a/documentation/docs/getting-started/configuration.md b/documentation/docs/getting-started/configuration.md index 39170b1..d569ef7 100644 --- a/documentation/docs/getting-started/configuration.md +++ b/documentation/docs/getting-started/configuration.md @@ -153,6 +153,8 @@ CORSConfig cors = CORSConfig( In production, set `origin` to your actual domain(s). Browsers reject `origin: '*'` together with `credentials: true`, so pick one. +A browser sends a cross-origin `QUERY` request only after a preflight that lists it, so add `QUERY` to `methods` if other origins call your [`QUERY` routes](../core/routing.md#query-routes). + ## File storage Storage is chosen by the `STORAGE` environment variable, not the config map. It is `local` by default; set it to `s3` to use AWS S3, and supply the S3 credentials via their own keys: @@ -180,6 +182,54 @@ APP_PRIVATE_KEY_PASSWORD= # only if the key is encrypted The server then binds with a `SecurityContext` built from those files. In most deployments you will instead terminate TLS at a reverse proxy or load balancer and leave the app on plain HTTP behind it. +## Server tuning + +```env +APP_ISOLATES=auto # isolates serving HTTP: a number, or auto for one per CPU core (default 1) +HTTP_IDLE_TIMEOUT=120 # seconds an idle keep-alive connection is kept; 0 keeps it forever +HTTP_BACKLOG=1024 # connections the OS queues while the server is busy +REQUEST_TIMEOUT=0 # seconds a request may take to start answering, upload included, before it gets a 503; 0 is off +SHUTDOWN_TIMEOUT=10 # seconds a shutdown waits for in-flight requests +SSE_HEARTBEAT=15 # seconds between keep-alive comments on a quiet event stream +MAX_BODY_SIZE=10485760 # largest request body in bytes, multipart uploads included +``` + +**`APP_ISOLATES`.** A single isolate runs on a single CPU core. With more than one, each isolate boots the app from the same config and they share the port, so throughput grows with the number of cores. Each isolate has its own memory, though, so anything kept in memory is per isolate: + +- **Rate limits** (`Throttle`) count per isolate, so a client gets up to that many times the configured allowance. +- **WebSocket rooms and broadcasts** reach only the clients connected to the same isolate. +- **The session cache** is switched off, and sessions are read from disk on every request so that no isolate serves a stale one. +- Anything your own code keeps in static fields or singletons. + +Keep the default of 1 if the app relies on any of these, or move that state somewhere shared, such as Redis. + +The extra isolates boot from the config but never run `main()`. Whatever `main()` does before `initialize` — registering a database driver, setting up auth — happens only in the first one, and the others serve without it. Pass that code as `setup`, which runs in every isolate before the providers boot: + +```dart +void main(List args) async { + await Application().initialize( + config: config, + args: args, + setup: () async { + registerMySqlDriver(); + await AuthServiceProvider().register(/* ... */); + }, + ); +} +``` + +`setup` is copied to each isolate, so it must not capture anything that cannot be, such as an open connection. + +An isolate that exits unexpectedly is replaced. One that keeps dying right after it starts, because its boot fails for instance, is retried less and less often, up to once a minute. + +**`HTTP_BACKLOG`** is how many incoming connections wait while the server is busy accepting; a burst beyond it is refused. The OS caps it at its own limit: `net.core.somaxconn` on Linux (usually 4096), `kern.ipc.somaxconn` on macOS (128 unless raised). + +**`REQUEST_TIMEOUT`** limits how long the app takes to start answering — a hung database call, say. Once the response has begun, it is left to finish: a stream of server-sent events or a large download is not cut off. + +The time it takes the client to send the request body counts too, since the app cannot answer before it has read it. That keeps a client trickling a body in byte by byte from holding a connection forever, but it also means a large upload over a slow connection is cut off with a 503. If the app takes uploads, set the timeout to what the largest one (`MAX_BODY_SIZE`) needs on the slowest connection you mean to serve, or leave it at 0. + +**Shutdown.** On `SIGINT` or `SIGTERM` the server stops accepting connections, lets in-flight requests finish for up to `SHUTDOWN_TIMEOUT`, writes out buffered log entries and exits. A second signal exits at once. + ## Reading config at runtime The `Config` singleton exposes whatever you put in the config map: diff --git a/documentation/docs/packages/swagger.md b/documentation/docs/packages/swagger.md index dc4244d..9c4c1d0 100644 --- a/documentation/docs/packages/swagger.md +++ b/documentation/docs/packages/swagger.md @@ -4,7 +4,9 @@ sidebar_position: 9 # Swagger / OpenAPI (vania_swagger) -The `vania_swagger` package generates OpenAPI 3.1.0 specifications from your routes and annotations, and serves a Swagger UI for interactive API exploration. +The `vania_swagger` package builds an OpenAPI 3.2.0 specification from descriptions you register in code, and serves it together with a Swagger UI for exploring and trying the API. + +The description is written by hand, endpoint by endpoint: the package does not read your routes or controllers. Keep each description next to the route it documents so the two change together. ## Installation @@ -15,26 +17,29 @@ dependencies: ## Setup -Register the service provider: +Add the service provider to the `providers` list of your config map: ```dart import 'package:vania_swagger/vania_swagger.dart'; -'providers': [ - RouteServiceProvider(), - SwaggerServiceProvider( - info: ApiInfo( - title: 'My App API', - version: '1.0.0', - description: 'REST API for My App', - contact: ApiContact(name: 'Dev Team', email: 'dev@example.com'), - license: ApiLicense(name: 'MIT'), +Map config = { + // ... + 'providers': [ + RouteServiceProvider(), + const SwaggerServiceProvider( + info: ApiInfo( + title: 'My App API', + version: '1.0.0', + description: 'REST API for My App', + contact: ApiContact(name: 'Dev Team', email: 'dev@example.com'), + license: ApiLicense(name: 'MIT'), + ), ), - ), -], + ], +}; ``` -This mounts three endpoints: +This adds three routes to the app's own router, on the app's own port: | Path | Description | |------|-------------| @@ -42,176 +47,264 @@ This mounts three endpoints: | `GET /docs/swagger.json` | OpenAPI spec (JSON) | | `GET /docs/swagger.yaml` | OpenAPI spec (YAML) | -## Describing Routes +### Production + +The docs list every endpoint, parameter and schema of the API, so they are **off when `APP_ENV` is `production`, and an app that does not set `APP_ENV` counts as production**. Set `APP_ENV=local` while developing, or `SWAGGER_ENABLED=true` to serve them anyway. If you do serve them outside development, put them behind authentication: + +```dart +SwaggerServiceProvider( + info: const ApiInfo(title: 'My App API', version: '1.0.0'), + middleware: [Authenticate()], +) +``` -### Using Helper Functions +## Describing Routes -The simplest way — no annotations or mirrors required: +Each HTTP method has a helper that adds one operation to the spec: `swaggerGet`, `swaggerPost`, `swaggerPut`, `swaggerPatch`, `swaggerDelete` and `swaggerQuery`. For any other method, use `swaggerRoute(method: ..., path: ...)`. Call them where you register the routes: ```dart -import 'package:vania_swagger/vania_swagger.dart'; +import 'package:vania/route.dart'; +import 'package:vania_swagger/vania_swagger.dart' hide ApiRoute; + +class ApiRoute extends Route { + @override + String? get prefix => 'api'; -class ApiRoute implements Route { @override void register() { - Router.basePrefix('api'); + super.register(); - swaggerRoute( - method: 'get', - path: '/api/users', - summary: 'List all users', + swaggerGet( + '/api/users', + operation: const ApiOperation(summary: 'List all users'), tags: ['Users'], - responses: { - 200: ApiOkResponse( + query: const [ApiQuery(name: 'page', schema: 'integer')], + responses: const [ + ApiOkResponse( description: 'A list of users', - content: {'type': 'array', 'items': {'\$ref': '#/components/schemas/User'}}, + rawSchema: { + 'type': 'array', + 'items': {r'$ref': '#/components/schemas/User'}, + }, ), - }, + ], ); Router.get('/users', userController.index); - swaggerRoute( - method: 'post', - path: '/api/users', - summary: 'Create a user', + swaggerPost( + '/api/users', + operation: const ApiOperation(summary: 'Create a user'), tags: ['Users'], - body: ApiBody( - required: true, - content: { - 'name': {'type': 'string'}, - 'email': {'type': 'string', 'format': 'email'}, - }, - ), - responses: { - 201: ApiCreatedResponse(description: 'User created'), - 422: ApiUnprocessableEntityResponse(description: 'Validation failed'), - }, + body: const ApiBody(schemaRef: 'User'), + responses: const [ + ApiCreatedResponse(description: 'User created', schemaRef: 'User'), + ApiUnprocessableEntityResponse(description: 'Validation failed'), + ], ); Router.post('/users', userController.store); + + swaggerGet( + '/api/users/{id}', + operation: const ApiOperation(summary: 'Get a user by ID'), + tags: ['Users'], + params: const [ApiParam(name: 'id', schema: 'integer')], + responses: const [ + ApiOkResponse(description: 'User found', schemaRef: 'User'), + ApiNotFoundResponse(description: 'User not found'), + ], + ); + Router.get('/users/{id}', userController.show).whereInt('id'); } } ``` -### Using Annotations +`vania_swagger` exports a class named `ApiRoute` too, so a route file named `ApiRoute`, as above, imports the package with `hide ApiRoute`. + +Paths in the spec are the full paths the client calls, prefix included. `/users/:id` is written as `/users/{id}` for you. + +### What an Operation Can Describe + +All the helpers take these named arguments, except that `swaggerDelete` has no `query`: + +| Argument | Type | Describes | +|----------|------|-----------| +| `operation` | `ApiOperation` | `summary`, `description`, `operationId`, `deprecated` | +| `tags` | `List` | The groups the operation is listed under | +| `params` | `List` | Path parameters (`location:` can also be `query`, `header` or `cookie`) | +| `query` | `List` | Query-string parameters | +| `headers` | `List` | Request headers | +| `responses` | `List` | Responses. Without any, a bare `200` is listed | +| `security` | `List` | The [security schemes](#security-schemes) it requires | +| `produces` | `List` | Response media types, `application/json` by default | +| `hidden` | `bool` | Leaves the operation out of the spec | + +`swaggerPost`, `swaggerPut`, `swaggerPatch` and `swaggerQuery` also take a request `body` (an `ApiBody`) and `consumes`, its media types. `swaggerPost` and `swaggerQuery` take `fileUploads` as well. + +A body or a response gets its schema from one of: `schemaRef: 'User'` (a [registered schema](#defining-schemas)), `schema: User` (a Dart type, referenced by its name), or `rawSchema:` (a JSON Schema map, used as is). -Annotate controller methods (requires `dart:mirrors`): +The response classes set the status code for you: `ApiOkResponse` (200), `ApiCreatedResponse` (201), `ApiNoContentResponse` (204), `ApiBadRequestResponse` (400), `ApiUnauthorizedResponse` (401), `ApiForbiddenResponse` (403), `ApiNotFoundResponse` (404) and `ApiUnprocessableEntityResponse` (422). For any other code, use `ApiResponse(statusCode: ...)`. + +### File Uploads ```dart -class UserController extends Controller { - @ApiOperation(summary: 'List all users', tags: ['Users']) - @ApiOkResponse(description: 'Success') - Future index() async { ... } - - @ApiOperation(summary: 'Create a user', tags: ['Users']) - @ApiBody(required: true) - @ApiCreatedResponse(description: 'User created') - @ApiUnprocessableEntityResponse(description: 'Validation failed') - Future store(Request req) async { ... } - - @ApiOperation(summary: 'Get a user by ID', tags: ['Users']) - @ApiParam(name: 'id', in_: 'path', required: true) - @ApiOkResponse(description: 'User found') - @ApiNotFoundResponse(description: 'User not found') - Future show(int id) async { ... } -} +swaggerPost( + '/api/users/{id}/avatar', + params: const [ApiParam(name: 'id', schema: 'integer')], + fileUploads: const [ + ApiFileUpload(name: 'avatar', allowedTypes: ['image/png', 'image/jpeg']), + ], +); ``` -Register annotated controllers with: +The body is documented as `multipart/form-data`, and Swagger UI shows a file picker. + +### QUERY Operations + +A [`QUERY` route](../core/routing.md#query-routes) is documented with `swaggerQuery`, which takes a request body like `swaggerPost`. List every media type the query may arrive in with `consumes`: ```dart -SwaggerModule.setup(controllers: [UserController], models: [User]); +swaggerQuery( + '/api/contacts', + operation: const ApiOperation(summary: 'Search contacts'), + body: const ApiBody(description: 'The query', rawSchema: {'type': 'object'}), + consumes: ['application/json', 'application/sql'], + tags: ['Contacts'], +); +Router.query('/contacts', contactController.search) + .acceptQuery(['application/json', 'application/sql']); ``` +It appears under the path's `query` field, which OpenAPI 3.2 added, and Swagger UI can send it from "Try it out". A method with no field of its own in OpenAPI, such as `PURGE`, is written under the path's `additionalOperations`, as 3.2 requires. + +### About the Annotations + +`ApiOperation`, `ApiParam`, `ApiOkResponse` and the other `Api*` classes have `const` constructors, so they can be written as annotations, and there are verb classes to go with them (`ApiGet`, `ApiPost`, `ApiPut`, `ApiPatch`, `ApiDelete`, and `ApiQueryMethod` for `QUERY`). **Nothing reads annotations, though.** That would take `dart:mirrors`, which ahead-of-time compiled apps cannot use, so an annotated controller method adds nothing to the spec. Pass the same objects to the helpers above instead. + ## Defining Schemas +Register a schema once, then refer to it by name with `schemaRef` wherever it is used: + +```dart +swaggerSchema( + 'User', + properties: const [ + ApiProperty(name: 'id', type: 'integer', required: true), + ApiProperty(name: 'name', type: 'string', required: true), + ApiProperty(name: 'email', type: 'string', format: 'email', required: true), + ApiProperty(name: 'created_at', type: 'string', format: 'date-time'), + ], +); +``` + +Or give the JSON Schema directly: + ```dart -swaggerSchema('User', { +swaggerRawSchema('User', { 'type': 'object', 'properties': { 'id': {'type': 'integer'}, 'name': {'type': 'string'}, 'email': {'type': 'string', 'format': 'email'}, - 'created_at': {'type': 'string', 'format': 'date-time'}, }, 'required': ['id', 'name', 'email'], }); ``` -Or use the annotation: - -```dart -@ApiSchema(properties: { - 'id': ApiProperty(type: 'integer'), - 'name': ApiProperty(type: 'string'), - 'email': ApiProperty(type: 'string', format: 'email'), -}) -class User extends Model { ... } -``` - ## Security Schemes +Declare a scheme once, then list it in the `security` of each operation that needs it. Swagger UI adds an "Authorize" button for it. + ### Bearer Token (JWT) ```dart -swaggerAddBearerAuth(); -``` - -Then reference it in route descriptions: +swaggerAddBearerAuth(); // registered as 'bearerAuth' -```dart -swaggerRoute( - method: 'get', - path: '/api/profile', - summary: 'Get current user profile', - security: [ApiSecurity(name: 'bearerAuth')], - // ... +swaggerGet( + '/api/profile', + operation: const ApiOperation(summary: 'Get the current user'), + security: const [ApiSecurity(scheme: 'bearerAuth')], ); ``` ### API Key ```dart -swaggerAddApiKey(name: 'X-API-Key', in_: 'header'); +swaggerAddApiKey(name: 'apiKey', headerName: 'X-API-Key'); ``` +`location:` places the key in the `ApiKeyLocation.header` (the default), `query` or `cookie`. + ### Basic Auth ```dart -swaggerAddBasicAuth(); +swaggerAddBasicAuth(); // registered as 'basicAuth' +``` + +### OAuth 2 + +```dart +swaggerAddOAuth2( + name: 'oauth', + authorizationUrl: 'https://auth.example.com/authorize', + tokenUrl: 'https://auth.example.com/token', + scopes: {'users:read': 'Read users'}, +); ``` -## Server Configuration +## Servers + +The spec always lists one server first: the `server_url` setting if there is one, otherwise the address the docs were requested on. `swaggerServer` adds more after it: ```dart -swaggerServer(url: 'https://api.example.com', description: 'Production'); -swaggerServer(url: 'http://localhost:8000', description: 'Local Development'); +swaggerServer('https://api.example.com', description: 'Production'); +swaggerServer('http://localhost:8000', description: 'Local development'); ``` ## Tags +Tags group operations in Swagger UI. Describing a tag is optional: + ```dart swaggerTag('Users', description: 'User management endpoints'); -swaggerTag('Posts', description: 'Blog post CRUD operations'); +swaggerTag( + 'Posts', + description: 'Blog post CRUD operations', + externalDocsUrl: 'https://example.com/docs/posts', +); ``` ## Configuration Options +Pass a `SwaggerConfig` to the provider: + ```dart -SwaggerServiceProvider( +const SwaggerServiceProvider( info: ApiInfo(title: 'My API', version: '1.0.0'), config: SwaggerConfig( enabled: true, basePath: '/docs', - serverUrl: 'http://localhost:8000', + title: 'My API Docs', + serverUrl: 'https://api.example.com', corsEnabled: true, ), -); +) ``` +Without one, the same settings come from the `swagger` entry of the config map, then from the environment: + +| Config key | Environment variable | Default | +|------------|----------------------|---------| +| `enabled` | `SWAGGER_ENABLED` | `true`, except in production | +| `base_path` | `SWAGGER_BASE_PATH` | `/docs` | +| `title` | `SWAGGER_TITLE` | `API Documentation` (the page title) | +| `server_url` | `SWAGGER_SERVER_URL` | the address the docs were requested on | +| `cors` | `SWAGGER_CORS` | `true`: the spec can be fetched from another origin | + ## Accessing the Spec Programmatically ```dart -Map spec = SwaggerGenerator().generate( - serverUrl: 'https://api.example.com', -); +final spec = SwaggerGenerator().generate(serverUrl: 'https://api.example.com'); +final yaml = encodeYaml(spec); ``` + +This is also how to test the docs: register the descriptions, generate the spec, and assert on it. Call `SwaggerMetadata().clear()` between tests, since every description goes into one shared registry. diff --git a/examples/core_showcase/.env b/examples/core_showcase/.env new file mode 100644 index 0000000..c64c34d --- /dev/null +++ b/examples/core_showcase/.env @@ -0,0 +1,15 @@ +APP_NAME=showcase +APP_ENV=local +APP_KEY=LX9cbQkiPf6EJWDKJmj45VkR+Ks76bdfxnUEgxkL+Ak= +APP_HOST=127.0.0.1 +APP_PORT=8790 +APP_DEBUG=false +# Plain HTTP on localhost: the session cookie must not be HTTPS-only. +SECURE_SESSION=false +CSRF_PROTECTION_ENABLED=true +MAX_BODY_SIZE=1048576 +REQUEST_TIMEOUT=3 +SSE_HEARTBEAT=2 +SHUTDOWN_TIMEOUT=5 +# Deliberately empty: an unset value must fall back to the default. +HTTP_BACKLOG= diff --git a/examples/core_showcase/.gitignore b/examples/core_showcase/.gitignore new file mode 100644 index 0000000..51eaa81 --- /dev/null +++ b/examples/core_showcase/.gitignore @@ -0,0 +1,3 @@ +.dart_tool/ +storage/ +pubspec.lock diff --git a/examples/core_showcase/README.md b/examples/core_showcase/README.md new file mode 100644 index 0000000..c2a70fa --- /dev/null +++ b/examples/core_showcase/README.md @@ -0,0 +1,11 @@ +# core_showcase + +Exercises the HTTP core end to end against the local `packages/core`. + +```bash +dart run bin/server.dart # http://127.0.0.1:8790/console runs the browser checks +dart run tool/check.dart # request-level checks against the running server +``` + +`tool/check.dart --isolates` covers `APP_ISOLATES` and graceful shutdown: it +starts and stops its own server. diff --git a/examples/core_showcase/bin/server.dart b/examples/core_showcase/bin/server.dart new file mode 100644 index 0000000..e0ba6c9 --- /dev/null +++ b/examples/core_showcase/bin/server.dart @@ -0,0 +1,6 @@ +import 'package:vania/vania.dart'; +import 'package:core_showcase/config/app.dart'; + +void main(List args) async { + await Application().initialize(config: config, args: args); +} diff --git a/examples/core_showcase/lib/app/middleware.dart b/examples/core_showcase/lib/app/middleware.dart new file mode 100644 index 0000000..0b10e5c --- /dev/null +++ b/examples/core_showcase/lib/app/middleware.dart @@ -0,0 +1,71 @@ +import 'package:vania/http/middleware.dart'; +import 'package:vania/http/request.dart'; +import 'package:vania/http/response.dart'; +import 'package:vania/vania.dart' show Unauthenticated, getSession; + +/// Refuses everything — guards `Router.any`, every method of it. +class DenyAll extends Middleware { + @override + Future handle(Request req) async => + throw Unauthenticated(message: 'Unauthenticated'); +} + +/// Adds a header after the controller has run. +class Timing extends Middleware { + @override + Future process(Request req, Next next) async { + final started = DateTime.now(); + await next(); + final ms = DateTime.now().difference(started).inMicroseconds / 1000; + req.response.headers.add('X-Duration-Ms', ms.toStringAsFixed(2)); + } +} + +/// A bug: runs the rest of the chain twice. +class CallsNextTwice extends Middleware { + @override + Future process(Request req, Next next) async { + await next(); + await next(); + } +} + +/// A bug: ends the chain without answering. +class Forgets extends Middleware { + @override + Future process(Request req, Next next) async {} +} + +/// Sees the controller's error on its way out, then lets it through. +class Observes extends Middleware { + @override + Future process(Request req, Next next) async { + try { + await next(); + } catch (e) { + req.response.headers.set('X-Observed-Error', e.runtimeType.toString()); + rethrow; + } + } +} + +/// Throws whatever [error] builds. +class Throws extends Middleware { + Throws(this.error); + final Object Function() error; + + @override + Future handle(Request req) async => throw error(); +} + +/// A session guard, as vania_auth's: a browser that is not signed in is +/// sent to the login page. +class SessionGuard extends Middleware { + @override + Future handle(Request req) async { + final signedIn = await getSession('logged_in') ?? false; + if (!signedIn) { + throw Unauthenticated(message: '/login', responseType: ResponseType.html); + } + } +} diff --git a/examples/core_showcase/lib/app/route_service_provider.dart b/examples/core_showcase/lib/app/route_service_provider.dart new file mode 100644 index 0000000..b24fb2f --- /dev/null +++ b/examples/core_showcase/lib/app/route_service_provider.dart @@ -0,0 +1,15 @@ +import 'package:vania/service_provider.dart'; + +import '../route/api_route.dart'; +import '../route/web_route.dart'; + +class RouteServiceProvider extends ServiceProvider { + @override + Future register() async {} + + @override + Future boot() async { + ApiRoute().register(); + WebRoute().register(); + } +} diff --git a/examples/core_showcase/lib/config/app.dart b/examples/core_showcase/lib/config/app.dart new file mode 100644 index 0000000..459f416 --- /dev/null +++ b/examples/core_showcase/lib/config/app.dart @@ -0,0 +1,12 @@ +import 'package:vania/service_provider.dart'; +import 'package:vania/vania.dart'; + +import '../app/route_service_provider.dart'; + +Map config = { + 'name': env('APP_NAME', 'showcase'), + 'url': env('APP_URL', 'http://127.0.0.1:8790'), + // `webhooks` covers /webhooks/stripe, not /webhooks-admin/purge. + 'csrf_except': ['webhooks'], + 'providers': [RouteServiceProvider()], +}; diff --git a/examples/core_showcase/lib/resources/view/console.html b/examples/core_showcase/lib/resources/view/console.html new file mode 100644 index 0000000..aab7d86 --- /dev/null +++ b/examples/core_showcase/lib/resources/view/console.html @@ -0,0 +1,217 @@ + + + + + + +Core showcase + + + +

Vania core — browser checks

+

Each row is a real request sent by this browser. Run again with the button.

+ +
running…
+
CheckResultDetail
+ +

Server-sent events, live

+
+ +

By hand

+ + + + + diff --git a/examples/core_showcase/lib/resources/view/dashboard.html b/examples/core_showcase/lib/resources/view/dashboard.html new file mode 100644 index 0000000..f02e28d --- /dev/null +++ b/examples/core_showcase/lib/resources/view/dashboard.html @@ -0,0 +1,7 @@ + +Dashboard + +

Dashboard

+

Signed in. Session id now: @{ sid }

+
{@ csrf @}
+ diff --git a/examples/core_showcase/lib/resources/view/errors/404.html b/examples/core_showcase/lib/resources/view/errors/404.html new file mode 100644 index 0000000..3d9fe1f --- /dev/null +++ b/examples/core_showcase/lib/resources/view/errors/404.html @@ -0,0 +1,3 @@ + +404 +

404

Nothing lives here.

(the app's own errors/404.html page)

diff --git a/examples/core_showcase/lib/resources/view/errors/419.html b/examples/core_showcase/lib/resources/view/errors/419.html new file mode 100644 index 0000000..454ca03 --- /dev/null +++ b/examples/core_showcase/lib/resources/view/errors/419.html @@ -0,0 +1,3 @@ + +419 +

419

The page expired — reload and try again.

(the app's own errors/419.html page)

diff --git a/examples/core_showcase/lib/resources/view/errors/500.html b/examples/core_showcase/lib/resources/view/errors/500.html new file mode 100644 index 0000000..8c9cd74 --- /dev/null +++ b/examples/core_showcase/lib/resources/view/errors/500.html @@ -0,0 +1,3 @@ + +500 +

500

Something broke on our side.

(the app's own errors/500.html page)

diff --git a/examples/core_showcase/lib/resources/view/form.html b/examples/core_showcase/lib/resources/view/form.html new file mode 100644 index 0000000..c1a3834 --- /dev/null +++ b/examples/core_showcase/lib/resources/view/form.html @@ -0,0 +1,16 @@ + +Form + +

Contact form

+

{@ session('status') @}

+
+ {@ csrf @} + +
{@ error('name') @}
+ +
{@ error('email') @}
+ + +
+

Back to the console

+ diff --git a/examples/core_showcase/lib/resources/view/login.html b/examples/core_showcase/lib/resources/view/login.html new file mode 100644 index 0000000..5fa4cff --- /dev/null +++ b/examples/core_showcase/lib/resources/view/login.html @@ -0,0 +1,10 @@ + +Login + +

Sign in

+

Session id before signing in: @{ sid }

+
+ {@ csrf @} + +
+ diff --git a/examples/core_showcase/lib/route/api_route.dart b/examples/core_showcase/lib/route/api_route.dart new file mode 100644 index 0000000..7055421 --- /dev/null +++ b/examples/core_showcase/lib/route/api_route.dart @@ -0,0 +1,209 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; +import 'dart:isolate'; +import 'dart:typed_data'; + +import 'package:vania/http/middleware.dart'; +import 'package:vania/http/request.dart'; +import 'package:vania/http/response.dart'; +import 'package:vania/route.dart'; +import 'package:vania/vania.dart' + show + InternalServerError, + NotFoundException, + Unauthenticated, + ValidationException; + +import '../app/middleware.dart'; + +/// JSON API routes, one block per feature under test. Everything here sits +/// under `api/`, which CSRF protection leaves out. +class ApiRoute extends Route { + @override + String? get prefix => 'api'; + + @override + void register() { + super.register(); + + // --- Basics, and a server that survives its handlers' failures. + Router.get('/json', () => Response.json({'ok': true})); + Router.get('/crash-after-write', (Request req) async { + req.response.write('partial'); + await req.response.flush(); + throw StateError('failed after the response started'); + }); + Router.get('/empty', () => Response()); + Router.get('/bad-json', () => Response.json({'when': DateTime(2020)})); + + // A bug in app code: a future nobody waits for fails after the + // response has gone. Nothing can catch it on the request's behalf. + Router.get('/stray-error', () { + unawaited( + Future.delayed( + const Duration(milliseconds: 20), + () => throw StateError('nobody awaited me'), + ), + ); + return Response.json({'answered': true}); + }); + + // --- Isolates, timeouts, shutdown. + Router.get( + '/who', + () => Response.json({'isolate': Isolate.current.debugName}), + ); + Router.get('/die', () { + if (Isolate.current.debugName?.startsWith('vania-http') ?? false) { + Isolate.exit(); + } + return Response.json({'isolate': 'main, not killed'}); + }); + Router.get('/sleep/{ms}', (int ms) async { + await Future.delayed(Duration(milliseconds: ms)); + return Response.json({'slept': ms}); + }); + + // --- Request body and accessors. + Router.post('/upload', (Request req) { + return Response.json({ + 'fields': req.all().keys.toList(), + 'avatarIsFile': req.file('avatar') != null, + }); + }); + Router.post('/accessors', (Request req) { + return Response.json({ + 'hasAge': req.has('age'), + 'exceptPassword': req.except('password'), + 'passwordStillThere': req.input('password'), + }); + }); + Router.get( + '/headers', + (Request req) => Response.json(req.headers['x-multi']), + ); + Router.post( + '/raw', + (Request req) => Response.json(utf8.decode(req.rawBody)), + ); + Router.post('/validate', (Request req) async { + await req.validate({'email': 'required|email'}); + return Response.json({'valid': true}); + }); + + // --- Files, ranges, downloads, events. + Router.get('/video', () => Response.fromPath('public/docs/sample.bin')); + Router.get( + '/download', + (Request req) => Response.fromPath( + 'public/docs/sample.bin', + download: true, + fileName: req.query('name') ?? 'sample.bin', + ), + ); + Router.get( + '/download-bytes', + (Request req) => Response.download( + req.query('name') ?? 'bytes.bin', + Uint8List.fromList(utf8.encode('downloaded')), + ), + ); + Router.get('/events', () { + return Response.sse( + Stream.periodic( + const Duration(milliseconds: 700), + (i) => {'tick': i + 1, 'at': DateTime.now().toIso8601String()}, + ).take(5), + ); + }); + + // --- Routing: constraints, decoding, any, apiResource. + Router.get('/slug/{s}', (String s) => Response.json(s)).whereString('s'); + Router.get('/price/{x}', (double x) => Response.json(x)).whereDouble('x'); + Router.get('/flag/{b}', (bool b) => Response.json(b)).whereBool('b'); + Router.get( + '/num/{id}', + (dynamic id) => Response.json(id), + ).where('id', r'\d+'); + Router.get( + '/posts/{slug}', + (String slug) => Response.json(slug), + ).whereString('slug'); + Router.get( + '/مقالات/{id}', + (int id) => Response.json({'article': id}), + ).whereInt('id'); + Router.any( + '/guarded', + () => Response.json({'secret': 42}), + ).middleware([DenyAll()]); + Router.apiResource('/photos', PhotoController()); + + // --- QUERY (RFC 10008). + Router.get('/contacts', () => Response.json({'via': 'GET'})); + Router.query('/contacts', (Request req) { + return Response.json({ + 'via': 'QUERY', + 'input': req.all(), + if (req.isJson() == false) 'raw': utf8.decode(req.rawBody), + }); + }).acceptQuery(['application/json', 'application/sql']); + + // --- Middleware and the response. + Router.get( + '/timed', + () => Response.json({'ok': true}), + ).middleware([Timing()]); + Router.get( + '/twice', + () => Response.json({'ok': true}), + ).middleware([CallsNextTwice()]); + Router.get( + '/forgets', + () => Response.json({'ok': true}), + ).middleware([Forgets()]); + Router.get( + '/observed', + () => throw StateError('controller failed'), + ).middleware([Observes()]); + Router.get('/throttled', () => Response.json({'ok': true})).middleware([ + // A short window, so the checker can be run again straight away. + Throttle( + maxAttempts: 3, + duration: const Duration(seconds: 3), + bypassInDevelopment: false, + ), + ]); + + // --- Errors: one policy wherever they are thrown. + Router.get( + '/missing-map', + () => throw NotFoundException(message: {'error': 'no such user'}), + ); + Router.get( + '/leaky', + () => throw InternalServerError(message: 'db password is hunter2'), + ); + Router.get('/leaky-mw', () => Response.json({})).middleware([ + Throws(() => InternalServerError(message: 'db password is hunter2')), + ]); + // A token guard: its message is text, so even a browser gets a 401. + Router.get( + '/token-guard', + () => Response.json({}), + ).middleware([Throws(() => Unauthenticated(message: 'Unauthenticated'))]); + Router.get('/validation-mw', () => Response.json({})).middleware([ + Throws(() => ValidationException(message: {'email': 'is required'})), + ]); + } +} + +/// A JSON resource: no `create` or `edit`. +class PhotoController { + Response index() => Response.json(['photo 1', 'photo 2']); + Response store() => Response.json({'stored': true}, HttpStatus.created); + Response show(int id) => Response.json({'photo': id}); + Response update(int id) => Response.json({'updated': id}); + Response destroy(int id) => Response.json({'destroyed': id}); +} diff --git a/examples/core_showcase/lib/route/web_route.dart b/examples/core_showcase/lib/route/web_route.dart new file mode 100644 index 0000000..d4cc43d --- /dev/null +++ b/examples/core_showcase/lib/route/web_route.dart @@ -0,0 +1,64 @@ +import 'package:vania/http/request.dart'; +import 'package:vania/http/response.dart'; +import 'package:vania/route.dart'; +import 'package:vania/vania.dart' + show + InternalServerError, + PageExpiredException, + destroyAllSessions, + regenerateSession, + setSession; + +import '../app/middleware.dart'; + +/// Pages: sessions, CSRF, forms, flash messages and error pages. +class WebRoute extends Route { + @override + void register() { + super.register(); + + // The page that runs the browser checks. + Router.get('/console', () => view('console')); + + // A form: CSRF token, validation errors, old input, a flash message. + Router.get('/form', () => view('form')); + Router.post('/form', (Request req) async { + await req.validate({'name': 'required', 'email': 'required|email'}); + return Response.back('status', 'Saved — thank you!'); + }); + + // Signing in moves the session to a new id. + Router.get( + '/login', + (Request req) => view('login', {'sid': _sessionId(req)}), + ); + Router.post('/login', (Request req) async { + await regenerateSession(); + await setSession('logged_in', true); + return Response.redirect('/dashboard'); + }); + Router.get( + '/dashboard', + (Request req) => view('dashboard', {'sid': _sessionId(req)}), + ).middleware([SessionGuard()]); + Router.post('/logout', () async { + await destroyAllSessions(); + return Response.redirect('/login'); + }); + + // State-changing routes behind CSRF protection. + Router.delete('/items/{id}', (int id) => Response.json({'deleted': id})); + Router.post('/webhooks/stripe', () => Response.json({'received': true})); + Router.post('/webhooks-admin/purge', () => Response.json({'purged': true})); + + // Error pages, each with its own status. + Router.get('/boom', () => throw InternalServerError(message: 'x')); + Router.get('/expired', () => throw PageExpiredException()); + + Router.get('/back', () => Response.back()); + } + + /// The session id the browser sent with this request. + static String _sessionId(Request req) => + req.cookie('showcase_session') ?? '(none yet)'; +} diff --git a/examples/core_showcase/public/.hidden b/examples/core_showcase/public/.hidden new file mode 100644 index 0000000..f3dbe9f --- /dev/null +++ b/examples/core_showcase/public/.hidden @@ -0,0 +1 @@ +APP_KEY=secret \ No newline at end of file diff --git a/examples/core_showcase/public/docs/sample.bin b/examples/core_showcase/public/docs/sample.bin new file mode 100644 index 0000000..5bab7ca Binary files /dev/null and b/examples/core_showcase/public/docs/sample.bin differ diff --git a/examples/core_showcase/public/index.html b/examples/core_showcase/public/index.html new file mode 100644 index 0000000..6ac8f35 --- /dev/null +++ b/examples/core_showcase/public/index.html @@ -0,0 +1,4 @@ + +Core showcase + +Open the console diff --git a/examples/core_showcase/public/my file.css b/examples/core_showcase/public/my file.css new file mode 100644 index 0000000..4487166 --- /dev/null +++ b/examples/core_showcase/public/my file.css @@ -0,0 +1 @@ +body{} /* spaced */ \ No newline at end of file diff --git "a/examples/core_showcase/public/\332\257\330\262\330\247\330\261\330\264.txt" "b/examples/core_showcase/public/\332\257\330\262\330\247\330\261\330\264.txt" new file mode 100644 index 0000000..618d168 --- /dev/null +++ "b/examples/core_showcase/public/\332\257\330\262\330\247\330\261\330\264.txt" @@ -0,0 +1 @@ +persian \ No newline at end of file diff --git a/examples/core_showcase/pubspec.yaml b/examples/core_showcase/pubspec.yaml new file mode 100644 index 0000000..5fcdc8b --- /dev/null +++ b/examples/core_showcase/pubspec.yaml @@ -0,0 +1,11 @@ +name: core_showcase +description: Exercises the HTTP core end to end — run it, then tool/check.dart. +version: 1.0.0 +publish_to: none + +environment: + sdk: ^3.12.2 + +dependencies: + vania: + path: ../../packages/core diff --git a/examples/core_showcase/secret.txt b/examples/core_showcase/secret.txt new file mode 100644 index 0000000..c61df65 --- /dev/null +++ b/examples/core_showcase/secret.txt @@ -0,0 +1 @@ +outside public \ No newline at end of file diff --git a/examples/core_showcase/tool/check.dart b/examples/core_showcase/tool/check.dart new file mode 100644 index 0000000..36a1dfd --- /dev/null +++ b/examples/core_showcase/tool/check.dart @@ -0,0 +1,746 @@ +// Request-level checks against a running core_showcase. +// +// dart run tool/check.dart # server already on :8790 +// dart run tool/check.dart --isolates # starts its own, with APP_ISOLATES +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; + +const host = '127.0.0.1'; +var port = 8790; +var passed = 0; +final failures = []; + +void report(String name, bool ok, [String detail = '']) { + if (ok) { + passed++; + } else { + failures.add(name); + } + print( + '${ok ? 'PASS' : 'FAIL'} $name${detail.isEmpty ? '' : ' — $detail'}', + ); +} + +Future check( + String name, + Future<(bool, String)> Function() body, { + Duration timeout = const Duration(seconds: 15), +}) async { + try { + final (ok, detail) = await body().timeout(timeout); + report(name, ok, detail); + } catch (e) { + report(name, false, 'threw $e'); + } +} + +/// A plain HTTP exchange, sent exactly as written. +class Raw { + Raw(this.status, this.headers, this.body); + final int status; + final Map> headers; + final String body; + String? header(String name) => headers[name.toLowerCase()]?.join(', '); +} + +Future raw( + String method, + String path, { + Map headers = const {}, + List extraHeaderLines = const [], + List? body, + bool chunked = false, +}) async { + final socket = await Socket.connect(host, port); + final lines = [ + '$method $path HTTP/1.1', + 'Host: $host:$port', + 'Connection: close', + for (final e in headers.entries) '${e.key}: ${e.value}', + ...extraHeaderLines, + if (body != null && !chunked) 'Content-Length: ${body.length}', + if (chunked) 'Transfer-Encoding: chunked', + ]; + // Read while writing: a server may answer — a 413, say — and close + // before the whole body has been sent. + final bytes = []; + final received = Completer(); + socket.listen( + bytes.addAll, + onError: (_) { + if (!received.isCompleted) received.complete(); + }, + onDone: () { + if (!received.isCompleted) received.complete(); + }, + cancelOnError: true, + ); + socket.done.catchError((_) {}); + try { + socket.add(utf8.encode('${lines.join('\r\n')}\r\n\r\n')); + if (body != null) { + if (chunked) { + for (var i = 0; i < body.length; i += 65536) { + // Stop once the server has answered, as curl does: writing on + // into a socket it has closed brings a reset that can wipe out + // the answer before it is read. + if (bytes.isNotEmpty) break; + final part = body.sublist(i, (i + 65536).clamp(0, body.length)); + socket.add(utf8.encode('${part.length.toRadixString(16)}\r\n')); + socket.add(part); + socket.add(utf8.encode('\r\n')); + await socket.flush(); + await Future.delayed(const Duration(milliseconds: 10)); + } + socket.add(utf8.encode('0\r\n\r\n')); + } else { + socket.add(body); + } + } + await socket.flush(); + } catch (_) { + // The server stopped reading; its answer is still on the way. + } + // An answer that never comes — a connection whose isolate has died — + // is no answer. + await received.future.timeout(const Duration(seconds: 10), onTimeout: () {}); + socket.destroy(); + final text = utf8.decode(bytes, allowMalformed: true); + final split = text.indexOf('\r\n\r\n'); + if (split < 0) return Raw(0, {}, ''); + final head = text.substring(0, split).split('\r\n'); + final headersOut = >{}; + for (final line in head.skip(1)) { + final colon = line.indexOf(':'); + headersOut + .putIfAbsent(line.substring(0, colon).toLowerCase(), () => []) + .add(line.substring(colon + 1).trim()); + } + var bodyText = text.substring(split + 4); + if (headersOut['transfer-encoding']?.contains('chunked') ?? false) { + bodyText = _unchunk(bodyText); + } + return Raw(int.parse(head.first.split(' ')[1]), headersOut, bodyText); +} + +String _unchunk(String s) { + final out = StringBuffer(); + var i = 0; + while (true) { + final eol = s.indexOf('\r\n', i); + if (eol < 0) break; + final size = int.tryParse(s.substring(i, eol).trim(), radix: 16) ?? 0; + if (size == 0) break; + out.write(s.substring(eol + 2, eol + 2 + size)); + i = eol + 2 + size + 2; + } + return out.toString(); +} + +/// A browser: keeps the cookies it is given. +class Browser { + final cookies = {}; + + String get cookieHeader => + cookies.entries.map((e) => '${e.key}=${e.value}').join('; '); + + Future send( + String method, + String path, { + String? form, + String accept = 'text/html', + String? referer, + }) async { + final res = await raw( + method, + path, + headers: { + 'Accept': accept, + if (cookies.isNotEmpty) 'Cookie': cookieHeader, + if (referer != null) 'Referer': referer, + if (form != null) 'Content-Type': 'application/x-www-form-urlencoded', + }, + body: form == null ? null : utf8.encode(form), + ); + for (final c in res.headers['set-cookie'] ?? const []) { + final pair = c.split(';').first; + final eq = pair.indexOf('='); + cookies[pair.substring(0, eq)] = pair.substring(eq + 1); + } + return res; + } + + static String? csrfIn(String html) => + RegExp(r'name="_csrf" value="([^"]+)"').firstMatch(html)?.group(1); +} + +const sessionCookie = 'showcase_session'; +String enc(Map form) => form.entries + .map((e) => '${e.key}=${Uri.encodeQueryComponent(e.value)}') + .join('&'); + +Future main(List args) async { + if (args.contains('--isolates')) { + await isolateChecks(); + } else { + await requestChecks(); + } + print('\n$passed passed, ${failures.length} failed'); + for (final f in failures) { + print(' FAILED: $f'); + } + exit(failures.isEmpty ? 0 : 1); +} + +Future requestChecks() async { + print('— stability'); + await check('server boots with HTTP_BACKLOG= left empty in .env', () async { + final r = await raw('GET', '/api/json'); + return (r.status == 200, 'status ${r.status}'); + }); + await check( + 'a failure after the response started does not stop the server', + () async { + try { + await raw('GET', '/api/crash-after-write'); + } catch (_) {} + final r = await raw('GET', '/api/json'); + return ( + r.status == 200 && r.body == '{"ok":true}', + 'next request: ${r.status}', + ); + }, + ); + await check( + 'a chunked multipart upload over MAX_BODY_SIZE is a 413', + () async { + final big = [ + ...utf8.encode( + '--B\r\nContent-Disposition: form-data; name="note"\r\n\r\n', + ), + ...List.filled(2 * 1024 * 1024, 65), + ...utf8.encode('\r\n--B--\r\n'), + ]; + final r = await raw( + 'POST', + '/api/upload', + headers: { + 'Content-Type': 'multipart/form-data; boundary=B', + 'Accept': 'application/json', + }, + body: big, + chunked: true, + ); + return (r.status == 413, 'status ${r.status}'); + }, + ); + await check( + 'an error app code leaves unhandled does not stop the server', + () async { + final r = await raw('GET', '/api/stray-error'); + await Future.delayed(const Duration(milliseconds: 200)); + final after = await raw('GET', '/api/json'); + return ( + r.status == 200 && after.status == 200, + 'then GET /api/json → ${after.status}', + ); + }, + ); + await check('REQUEST_TIMEOUT answers a hung request with a 503', () async { + final sw = Stopwatch()..start(); + final r = await raw( + 'GET', + '/api/sleep/6000', + headers: {'Accept': 'application/json'}, + ); + return ( + r.status == 503 && sw.elapsed.inMilliseconds < 4500, + 'status ${r.status} after ${sw.elapsed.inMilliseconds} ms', + ); + }); + + print('— request and response'); + await check('multipart: fields read, a text field is not a file', () async { + final body = utf8.encode( + '--B\r\nContent-Disposition: form-data; name="avatar"\r\n\r\nnot a file\r\n' + '--B\r\nContent-Disposition: form-data; name="doc"; filename="a.txt"\r\nContent-Type: text/plain\r\n\r\nhello\r\n--B--\r\n', + ); + final r = await raw( + 'POST', + '/api/upload', + headers: { + 'Content-Type': 'multipart/form-data; boundary=B', + 'Accept': 'application/json', + }, + body: body, + ); + final b = jsonDecode(r.body); + return ( + r.status == 200 && + b['avatarIsFile'] == false && + (b['fields'] as List).contains('doc'), + r.body, + ); + }); + await check('has() on a number, except() leaves the input alone', () async { + final r = await raw( + 'POST', + '/api/accessors', + headers: { + 'Content-Type': 'application/json', + 'Accept': 'application/json', + }, + body: utf8.encode('{"age":30,"user":"a","password":"x"}'), + ); + final b = jsonDecode(r.body); + return ( + b['hasAge'] == true && + !(b['exceptPassword'] as Map).containsKey('password') && + b['passwordStillThere'] == 'x', + r.body, + ); + }); + await check('a header sent on two lines reads "a, b"', () async { + final r = await raw( + 'GET', + '/api/headers', + headers: {'Accept': 'application/json'}, + extraHeaderLines: ['X-Multi: a', 'X-Multi: b'], + ); + return (r.body == '"a, b"', r.body); + }); + await check('Request.rawBody', () async { + final r = await raw( + 'POST', + '/api/raw', + headers: {'Content-Type': 'text/plain', 'Accept': 'application/json'}, + body: utf8.encode('just text'), + ); + return (r.body == '"just text"', r.body); + }); + await check('Response() with no data has an empty body', () async { + final r = await raw('GET', '/api/empty'); + return ( + r.status == 200 && r.body.isEmpty, + 'status ${r.status}, body "${r.body}"', + ); + }); + await check('an unencodable JSON response is a 500', () async { + final r = await raw('GET', '/api/bad-json'); + return ( + r.status == 500 && r.body == '{"message":"Internal Server Error"}', + '${r.status} ${r.body}', + ); + }); + await check('SSE: every event arrives, even past REQUEST_TIMEOUT', () async { + // 5 events 700 ms apart: 3.5 s, past the sample's 3 s REQUEST_TIMEOUT. + final r = await raw('GET', '/api/events'); + final ticks = RegExp( + r'"tick":(\d+)', + ).allMatches(r.body).map((m) => m.group(1)).join(','); + return (ticks == '1,2,3,4,5', 'ticks $ticks'); + }); + await check('SSE: the first event arrives at once, not at the end', () async { + final socket = await Socket.connect(host, port); + socket.write( + 'GET /api/events HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n', + ); + final sw = Stopwatch()..start(); + final firstAt = Completer(); + final sub = socket.listen((d) { + if (!firstAt.isCompleted && + utf8.decode(d, allowMalformed: true).contains('data:')) { + firstAt.complete(sw.elapsedMilliseconds); + } + }); + final ms = await firstAt.future; + await sub.cancel(); + socket.destroy(); + return (ms < 1500, 'first event after $ms ms (5 events, 700 ms apart)'); + }); + + print('— routing'); + final routes = <(String, int, String?)>[ + ('/api/slug/2024', 200, '"2024"'), + ('/api/price/5', 200, '5.0'), + ('/api/price/abc', 404, null), + ('/api/flag/TRUE', 200, 'true'), + ('/api/flag/yes', 404, null), + ('/api/num/123', 200, '123'), + ('/api/num/abc123', 404, null), + ('/api/posts/c++', 200, '"c++"'), + ('/api/posts/a%2Fb', 200, '"a/b"'), + ('/api/posts/%D8%B3%D9%84%D8%A7%D9%85', 200, '"سلام"'), + ('/api/posts/%E0%A4%A', 404, null), + ('/api/${Uri.encodeComponent('مقالات')}/7', 200, '{"article":7}'), + ]; + for (final (path, status, body) in routes) { + await check('GET $path → $status${body == null ? '' : ' $body'}', () async { + final r = await raw('GET', path, headers: {'Accept': 'application/json'}); + return ( + r.status == status && (body == null || r.body == body), + '${r.status} ${r.body}', + ); + }); + } + await check( + 'apiResource: index, store, show, update (PUT + PATCH), destroy', + () async { + final got = []; + for (final (m, p) in [ + ('GET', '/api/photos'), + ('POST', '/api/photos'), + ('GET', '/api/photos/3'), + ('PUT', '/api/photos/3'), + ('PATCH', '/api/photos/3'), + ('DELETE', '/api/photos/3'), + ]) { + got.add( + '${(await raw(m, p, headers: {'Accept': 'application/json'})).status}', + ); + } + return (got.join(',') == '200,201,200,200,200,200', got.join(',')); + }, + ); + await check('apiResource has no create or edit page', () async { + final a = (await raw('GET', '/api/photos/create')).status; + final b = (await raw('GET', '/api/photos/3/edit')).status; + return (a == 404 && b == 404, '$a, $b'); + }); + await check('HEAD on a static file: headers only', () async { + final r = await raw('HEAD', '/my%20file.css'); + return ( + r.status == 200 && r.body.isEmpty && r.header('content-length') == '19', + 'status ${r.status}, length ${r.header('content-length')}', + ); + }); + + print('— static files never leak'); + for (final path in [ + '/%2E%2E/secret.txt', + '/docs/%2E%2E/%2E%2E/secret.txt', + '/docs%2F..%2F..%2Fsecret.txt', + '/..%5Csecret.txt', + '/%2Ehidden', + '/docs/%2E%2E/.hidden', + ]) { + await check('GET $path stays inside public/', () async { + final r = await raw('GET', path); + return ( + r.status != 200 && + !r.body.contains('outside public') && + !r.body.contains('APP_KEY'), + 'status ${r.status}', + ); + }); + } + + print('— middleware'); + await check( + 'calling next() twice is a 500, not a second controller run', + () async { + final r = await raw( + 'GET', + '/api/twice', + headers: {'Accept': 'application/json'}, + ); + return (r.status == 500, 'status ${r.status}'); + }, + ); + await check('a middleware that forgets to answer: 500, not a hang', () async { + final r = await raw( + 'GET', + '/api/forgets', + headers: {'Accept': 'application/json'}, + ); + return (r.status == 500, 'status ${r.status}'); + }); + await check( + 'Throttle: 200 ×3 then 429; Remaining 2,1,0,0; one Reset', + () async { + // The window is 3 s; a run just before this one may still be in it. + await Future.delayed(const Duration(milliseconds: 3100)); + final statuses = [], remaining = [], resets = {}; + String? retry; + for (var i = 0; i < 4; i++) { + final r = await raw( + 'GET', + '/api/throttled', + headers: {'Accept': 'application/json'}, + ); + statuses.add(r.status); + remaining.add(r.header('x-ratelimit-remaining')); + resets.add(r.header('x-ratelimit-reset')); + retry = r.header('retry-after') ?? retry; + } + return ( + statuses.join(',') == '200,200,200,429' && + remaining.join(',') == '2,1,0,0' && + resets.length == 1 && + retry != null, + 'statuses $statuses, remaining $remaining, resets ${resets.length}, Retry-After $retry', + ); + }, + ); + + print('— errors'); + await check('NotFoundException with a map keeps 404 and its body', () async { + final r = await raw( + 'GET', + '/api/missing-map', + headers: {'Accept': 'application/json'}, + ); + return ( + r.status == 404 && r.body == '{"error":"no such user"}', + '${r.status} ${r.body}', + ); + }); + await check( + 'a ValidationException from middleware: 422 for an API', + () async { + final r = await raw( + 'GET', + '/api/validation-mw', + headers: {'Accept': 'application/json'}, + ); + return (r.status == 422, '${r.status} ${r.body}'); + }, + ); + await check('validate() in a controller: 422 with the errors', () async { + final r = await raw( + 'POST', + '/api/validate', + headers: { + 'Content-Type': 'application/json', + 'Accept': 'application/json', + }, + body: utf8.encode('{}'), + ); + return ( + r.status == 422 && r.body.contains('email'), + '${r.status} ${r.body}', + ); + }); + + print('— sessions, CSRF, flash'); + await check('an id the server never issued is not taken up', () async { + final r = await raw( + 'GET', + '/login', + headers: { + 'Accept': 'text/html', + 'Cookie': '$sessionCookie=chosen-by-attacker', + }, + ); + final issued = (r.headers['set-cookie'] ?? []).firstWhere( + (c) => c.startsWith('$sessionCookie='), + orElse: () => '', + ); + return ( + issued.isNotEmpty && !issued.contains('chosen-by-attacker'), + issued.split(';').first, + ); + }); + await check( + 'signing in moves the session to a new id; the old one ends', + () async { + final b = Browser(); + final page = await b.send('GET', '/login'); + final before = b.cookies[sessionCookie]; + final token = Browser.csrfIn(page.body); + final login = await b.send( + 'POST', + '/login', + form: enc({'_csrf': token ?? ''}), + ); + final after = b.cookies[sessionCookie]; + final dash = await b.send('GET', '/dashboard'); + final old = await raw( + 'GET', + '/dashboard', + headers: {'Accept': 'text/html', 'Cookie': '$sessionCookie=$before'}, + ); + final ok = + login.status == 302 && + login.header('location') == '/dashboard' && + before != after && + dash.status == 200 && + old.status == 302; + return ( + ok, + 'login ${login.status} → ${login.header('location')}, id changed: ${before != after}, dashboard ${dash.status}, old id → ${old.status} ${old.header('location')}', + ); + }, + ); + await check('a form post without the CSRF token is a 419 page', () async { + final b = Browser(); + await b.send('GET', '/form'); + final r = await b.send( + 'POST', + '/form', + form: enc({'name': 'Ann', 'email': 'a@b.c'}), + ); + return ( + r.status == 419 && r.body.contains("the app's own errors/419"), + 'status ${r.status}', + ); + }); + await check( + 'validation errors and old input survive the redirect, once; no password', + () async { + final b = Browser(); + final page = await b.send('GET', '/form'); + final token = Browser.csrfIn(page.body)!; + final post = await b.send( + 'POST', + '/form', + referer: 'http://$host:$port/form', + form: enc({ + '_csrf': token, + 'name': 'Ann', + 'email': 'not-an-email', + 'password': 'secret123', + }), + ); + final back = await b.send('GET', '/form'); + final again = await b.send('GET', '/form'); + final errorShown = RegExp(r'id="email-error">[^<]+<').hasMatch(back.body); + final oldName = back.body.contains('value="Ann"'); + final noPassword = !back.body.contains('secret123'); + final gone = !RegExp(r'id="email-error">[^<]+<').hasMatch(again.body); + return ( + post.status == 302 && + post.header('location') == '/form' && + errorShown && + oldName && + noPassword && + gone, + 'POST ${post.status} → ${post.header('location')}, error shown $errorShown, old name $oldName, password hidden $noPassword, gone next time $gone', + ); + }, + ); + await check('a flash message reaches the next page, once', () async { + final b = Browser(); + final page = await b.send('GET', '/form'); + final token = Browser.csrfIn(page.body)!; + await b.send( + 'POST', + '/form', + referer: 'http://$host:$port/form', + form: enc({'_csrf': token, 'name': 'Ann', 'email': 'ann@example.com'}), + ); + final next = await b.send('GET', '/form'); + final after = await b.send('GET', '/form'); + final shown = next.body.contains('Saved — thank you!'); + final gone = !after.body.contains('Saved — thank you!'); + return (shown && gone, 'shown $shown, gone after $gone'); + }); + await check( + 'back() follows a same-site Referer and ignores another site', + () async { + final same = await raw( + 'GET', + '/back', + headers: { + 'Accept': 'text/html', + 'Referer': 'http://$host:$port/form?x=1', + }, + ); + final evil = await raw( + 'GET', + '/back', + headers: { + 'Accept': 'text/html', + 'Referer': 'https://evil.example/phish', + }, + ); + return ( + same.header('location') == '/form?x=1' && + evil.header('location') == '/', + 'same-site → ${same.header('location')}, other → ${evil.header('location')}', + ); + }, + ); +} + +Future isolateChecks() async { + port = 8791; + print('— APP_ISOLATES=3, graceful shutdown (server on :$port)'); + final server = await Process.start( + 'dart', + ['run', 'bin/server.dart', '--port', '$port'], + environment: {'APP_ISOLATES': '3'}, + ); + server.stdout.drain(); + server.stderr.drain(); + for (var i = 0; i < 60; i++) { + try { + if ((await raw('GET', '/api/json')).status == 200) break; + } catch (_) {} + await Future.delayed(const Duration(milliseconds: 500)); + } + + Future> isolates() async { + final seen = {}; + for (var i = 0; i < 60; i++) { + seen.add( + jsonDecode((await raw('GET', '/api/who')).body)['isolate'].toString(), + ); + } + return seen; + } + + await check('requests are spread over three isolates', () async { + final seen = await isolates(); + return (seen.length == 3, '$seen'); + }); + await check('a stray error in a server isolate does not stop it', () async { + for (var i = 0; i < 10; i++) { + await raw('GET', '/api/stray-error'); + } + await Future.delayed(const Duration(milliseconds: 300)); + final seen = await isolates(); + return (seen.length == 3, '$seen'); + }); + await check('a server isolate that dies is replaced', () async { + for (var i = 0; i < 6; i++) { + try { + await raw('GET', '/api/die'); + } catch (_) {} + } + // An isolate that keeps dying soon after it starts is retried less + // and less often (1 s, 2 s, 4 s …), so this can take a while. + final waited = Stopwatch()..start(); + var seen = {}; + while (waited.elapsed < const Duration(seconds: 50)) { + seen = await isolates(); + if (seen.length == 3) break; + await Future.delayed(const Duration(seconds: 2)); + } + return (seen.length == 3, '$seen after ${waited.elapsed.inSeconds} s'); + }, timeout: const Duration(seconds: 60)); + await check( + 'SIGTERM: an in-flight request finishes, then the process exits', + () async { + final inFlight = raw('GET', '/api/sleep/2000'); + await Future.delayed(const Duration(milliseconds: 400)); + server.kill(ProcessSignal.sigterm); + await Future.delayed(const Duration(milliseconds: 300)); + // Not served: refused outright, or accepted while closing and left + // unanswered. + var refused = false; + try { + refused = (await raw('GET', '/api/json')).status == 0; + } catch (_) { + refused = true; + } + final r = await inFlight; + final code = await server.exitCode.timeout(const Duration(seconds: 10)); + return ( + r.status == 200 && refused && code == 0, + 'in-flight ${r.status}, new connection refused $refused, exit code $code', + ); + }, + ); +} diff --git a/packages/core/.gitignore b/packages/core/.gitignore index c0ff9b9..e484a68 100644 --- a/packages/core/.gitignore +++ b/packages/core/.gitignore @@ -7,3 +7,6 @@ # Avoid committing pubspec.lock for library packages; see # https://dart.dev/guides/libraries/private-files#pubspeclock. pubspec.lock + +# Sessions and logs written by the tests. +/storage/ diff --git a/packages/core/CHANGELOG.md b/packages/core/CHANGELOG.md index 9e8cf25..c84de8d 100644 --- a/packages/core/CHANGELOG.md +++ b/packages/core/CHANGELOG.md @@ -1,3 +1,125 @@ +## 2.1.0 + +This release is mostly about the HTTP layer: what happens between a request +arriving and the response leaving. It is faster, it no longer falls over when +a handler misbehaves, and a few long-standing bugs around CSRF, sessions and +flash messages are gone. It also adds the new HTTP `QUERY` method. + +Vania is API-first, and this release leans into that: JSON with a real status +code is the default everywhere, and redirects or HTML pages only happen when +the app actually has views or asks for them. + +### Things that behave differently + +- **The response is written after the middleware chain.** Your controller + returns a `Response` and Vania writes it once every middleware is done, so + a middleware can still add headers after `await next()`. If the controller + throws, the error now comes back out of `await next()` where a middleware + can see it. Calling `next()` twice throws a `StateError` instead of running + the controller a second time. +- **Errors get one answer, wherever they're thrown.** Throwing from a + middleware or from a controller gives the same response. An app with no + views answers in JSON even if a browser asks for HTML. The message of a 5xx + is only shown when `APP_DEBUG` is on. `Unauthenticated` only redirects to a + login page when you create it with `responseType: ResponseType.html`; + otherwise it's a plain 401. +- **`where()` matches the whole segment.** `r'\d+'` used to accept `abc123`; + it doesn't anymore. `whereString`, `whereDouble` and `whereBool` are + actually enforced now, and the controller gets the declared type. +- **Paths are decoded before matching.** Route parameters reach the + controller decoded (`سلام دنیا`, not `%D8%B3...`), and a route written in + Persian or any other script matches. +- **CSRF** runs before any other middleware and now covers `DELETE` too. An + entry in `csrf_except` covers that path and what's under it, so `webhooks` + no longer accidentally covers `webhooks-admin`. A `.` in a wildcard pattern + means a literal dot. +- **Sessions.** A session id the server never handed out is ignored instead + of adopted, by page requests and API calls alike. The stored session lives for `SESSION_LIFETIME` (it used to be + a fixed hour) and gets extended once it's past half of that. Changes are + saved once, right before the response, and only the keys that changed are + written. +- `Router.any(...).middleware(...)` (and `.where()`, `.name()` and friends) + now applies to every method `any` registered, not just the last one. Inside + a group, `any` also picks up the group's prefix. +- A response whose data can't be encoded as JSON is a 500, not a 200 with a + broken body. `Response()` with no data sends an empty body instead of + `null`. +- **`Throttle` no longer turns itself off in development** unless you ask + with `bypassInDevelopment: true`; then it's skipped while `APP_ENV` is + `local` or `development`. A production app running on a `.env` copied + from a laptop kept no rate limits before. +- `env`, `env` and the other typed reads treat an empty value + (`KEY=`) as not set and return your default. +- `ROUTE_LOOK_UP_SIZE` is gone, along with the route cache it sized. Route + lookup is indexed now and doesn't need it. + +### New + +- Serve from several isolates with `APP_ISOLATES` (a number, or `auto`). If + one crashes, a new one takes its place. The extra isolates don't run + `main()`, so move what you do there before `initialize` — registering a + database driver, setting up auth — into the new `setup:` callback, which + runs in every isolate: + + ```dart + await Application().initialize( + config: config, + setup: () => registerMySqlDriver(), + ); + ``` +- Graceful shutdown on `SIGINT`/`SIGTERM`: requests in flight get up to + `SHUTDOWN_TIMEOUT` seconds to finish. +- `REQUEST_TIMEOUT`, `HTTP_IDLE_TIMEOUT` and `HTTP_BACKLOG` settings. +- The HTTP `QUERY` method (RFC 10008): `Router.query(...)`, + `.acceptQuery([...])` to say which content types a route takes, the + `Accept-Query` header, and an `Allow` header on `OPTIONS`. +- `HEAD` requests are answered by the matching `GET` route. +- `Request.rawBody` for bodies that aren't JSON or forms. +- `Response.fromPath(...)` streams a file from disk and supports `Range`, so + large downloads and video seeking don't load the file into memory. +- `Router.apiResource(...)`: a resource without the `create`/`edit` form + pages. +- `regenerateSession()` to give a session a new id, e.g. right after login. +- `Logger.flush()`. +- Server-sent events are actually delivered as they happen now, with a + heartbeat (`SSE_HEARTBEAT`), and the stream is cancelled when the client + goes away. + +### Fixed + +- An error thrown after the response had started could take the whole server + down. It doesn't anymore, and neither does a stray future that fails with + nobody awaiting it. +- With CSRF protection on, every `POST` was rejected because the cookie was + read before it was parsed. +- Flash messages, validation errors and old input now survive the redirect. +- HTML error responses and the `errors/*.html` pages are sent with the real + error status instead of 200. +- Downloads with a non-ASCII name (`گزارش.pdf`) failed with a 500. They now + use `filename*` so browsers save them under the right name. +- Static files with spaces or non-Latin letters in their name weren't found. +- `MAX_BODY_SIZE` is enforced for chunked multipart uploads as well. +- Session files are written atomically, and a damaged one is treated as no + session rather than an error. +- `Response.back()` only follows a same-site `Referer` and no longer loops on + `GET`. +- `Request.has`, `except`, `headers`, `file` and `files` misbehaved on some + inputs. +- `Throttle` reports `X-RateLimit-Remaining` after counting the request and + sends the real reset time. +- Two routes with the same path, one for a domain and one for any host, + overwrote each other; now both work and the host decides. An encoded `/` + (`a%2Fb`) no longer finds the static route `a/b`. +- The `XSRF-TOKEN` cookie expired after a fixed 2.5 hours, so a form left + open longer got a 419 while its session was still alive. It now lasts as + long as the session and is renewed with it. +- A server isolate that keeps failing to start is retried at most once a + minute, instead of the wait starting over at one second. +- A middleware that neither calls `next()` nor returns a response now gets a + 500 instead of leaving the request hanging. +- A `Router.group` callback that throws no longer leaks its prefix and + middleware into the routes registered after it. + ## 2.0.0 Vania 2.0 makes the framework driver-agnostic. Everything portable — the ORM, diff --git a/packages/core/LICENSE b/packages/core/LICENSE new file mode 100644 index 0000000..ddb3578 --- /dev/null +++ b/packages/core/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2024 Vania + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/core/lib/application.dart b/packages/core/lib/application.dart index 4851faf..c74c09a 100644 --- a/packages/core/lib/application.dart +++ b/packages/core/lib/application.dart @@ -1,12 +1,24 @@ +import 'dart:async'; +import 'dart:io'; +import 'dart:isolate'; + import 'src/container.dart'; import 'src/ioc_container.dart'; import 'src/localization_handler/localization.dart'; +import 'src/logger/logger.dart'; +import 'src/route/route_inspector.dart' show routeDumpEnvVar; import 'src/server/base_http_server.dart'; +import 'src/server/server_isolates.dart'; +import 'src/server/uncaught_errors.dart'; import 'src/env_handler/env.dart'; import 'src/http/request/request_handler.dart'; import 'src/http/session/session_manager.dart'; import 'env.dart' show env; +/// Set in the extra isolates started for `APP_ISOLATES`. Each isolate has +/// its own copy, so it stays false in the main one. +bool _isWorker = false; + class Application extends Container { static Application? _singleton; @@ -20,12 +32,44 @@ class Application extends Container { Application._internal(); - late BaseHttpServer _server; + BaseHttpServer? _server; + + final List<_Worker> _workers = <_Worker>[]; + /// A copy of the config and setup taken before this isolate's providers + /// boot, for starting a replacement worker later: by then the original + /// may hold things that cannot be sent to another isolate, such as a + /// connection. + late final _WorkerBoot _workerBoot; + + Future? _closing; + + /// Boots the app and starts serving. + /// + /// [setup] runs once in every isolate that serves requests, before the + /// config's providers boot. Put there what would otherwise sit in + /// `main()` ahead of this call — registering a database driver, an auth + /// provider — because with `APP_ISOLATES` above 1 each extra isolate + /// starts from scratch and never runs `main()`: + /// + /// ```dart + /// await Application().initialize( + /// config: config, + /// setup: () async { + /// registerMySqlDriver(); + /// await AuthServiceProvider().register(...); + /// }, + /// ); + /// ``` + /// + /// It is sent to each isolate, so it must not capture anything that + /// cannot be: an open connection or socket, say. Future initialize({ required Map config, List args = const [], + FutureOr Function()? setup, }) async { + keepServingOnUncaughtErrors(log: !_isWorker); await Localization().init(); IoCContainer().register(() => RequestHandler()); @@ -44,11 +88,196 @@ class Application extends Container { ); } - _server = BaseHttpServer(config: config, args: args); - _server.startServer(); + // Here first, so a setup that fails stops the app before any other + // isolate has started. + await setup?.call(); + + // Workers get the config before this isolate's providers boot, so + // each one boots from the same untouched config. + if (!_isWorker && + serverIsolateCount > 1 && + Platform.environment[routeDumpEnvVar] != '1') { + await _spawnWorkers(config, args, setup); + } + + final server = BaseHttpServer( + config: config, + args: args, + isWorker: _isWorker, + ); + _server = server; + await server.startServer(); + + if (!_isWorker) _closeOnSignals(); + } + + /// Stops every isolate's server, letting in-flight requests finish for + /// up to `SHUTDOWN_TIMEOUT`, then writes out buffered log entries. + Future close() => _closing ??= _close(); + + Future _close() async { + final timeout = shutdownTimeout; + for (final worker in _workers) { + worker.restart?.cancel(); + worker.control?.send(null); + } + await Future.wait([ + if (_server != null) _server!.shutdown(timeout: timeout), + // A worker that does not answer is left to the process exit. + for (final worker in _workers) + worker.exited.future.timeout( + timeout + const Duration(seconds: 1), + onTimeout: () {}, + ), + ]); + await Logger.flush(); + } + + Future _spawnWorkers( + Map config, + List args, + FutureOr Function()? setup, + ) async { + // Sending a message copies it, even within one isolate. + final copy = ReceivePort(); + copy.sendPort.send((config, args, setup)); + _workerBoot = await copy.first as _WorkerBoot; + copy.close(); + + for (var i = 1; i < serverIsolateCount; i++) { + final worker = await _spawnWorker(i, const Duration(seconds: 1)); + if (worker == null) return; + _workers.add(worker); + } + } + + /// Starts the server isolate for slot [index], or returns null when it + /// cannot be started. [retryAfter] is how long to wait before replacing + /// it should it exit. + Future<_Worker?> _spawnWorker(int index, Duration retryAfter) async { + final worker = _Worker(index); + try { + await Isolate.spawn( + _workerMain, + (_workerBoot, worker.ready.sendPort), + onExit: worker.exitPort.sendPort, + onError: worker.errors.sendPort, + debugName: 'vania-http-$index', + ); + } catch (e) { + // Most likely the config or the setup holds something that cannot be + // copied to another isolate, which every further spawn would hit as + // well. + worker.dispose(); + Logger.log( + 'APP_ISOLATES: could not start a server isolate ($e); ' + 'serving with ${_workers.length + 1}.', + type: Logger.WARNING, + ); + return null; + } + worker.listen(onExit: () => _replace(worker, retryAfter)); + return worker; + } + + /// Starts a new isolate in place of [worker], which exited without being + /// asked to, after the wait [isolateRestartDelay] gives. + void _replace(_Worker worker, Duration retryAfter) { + if (_closing != null) return; + final (:wait, :next) = isolateRestartDelay( + retryAfter, + crashedEarly: worker.uptime.elapsed < const Duration(seconds: 10), + ); + Logger.log( + 'Server isolate vania-http-${worker.index} exited; ' + 'starting a new one in ${wait.inSeconds}s.', + type: Logger.ERROR, + ); + worker.restart = Timer(wait, () async { + if (_closing != null) return; + final replacement = await _spawnWorker(worker.index, next); + if (replacement == null) return; + final slot = _workers.indexOf(worker); + if (slot == -1) return; + _workers[slot] = replacement; + }); + } + + void _closeOnSignals() { + final signals = [ + ProcessSignal.sigint, + if (!Platform.isWindows) ProcessSignal.sigterm, + ]; + for (final signal in signals) { + signal.watch().listen((_) async { + // A second signal while draining means: stop now. + if (_closing != null) exit(1); + await close(); + exit(0); + }); + } + } +} + +/// A server isolate as seen from the main one. +class _Worker { + /// Its slot, kept by any replacement. + final int index; + + final Stopwatch uptime = Stopwatch()..start(); + + final ReceivePort ready = ReceivePort(); + final ReceivePort exitPort = ReceivePort(); + final ReceivePort errors = ReceivePort(); + final Completer exited = Completer(); + + /// Where the shutdown request is sent, once the isolate is serving. + SendPort? control; + + /// The pending start of a replacement, after this one exited. + Timer? restart; + + _Worker(this.index); + + void listen({required void Function() onExit}) { + ready.listen((port) => control = port as SendPort); + errors.listen((error) { + final [message, stack] = error as List; + Logger.log('Server isolate error: $message\n$stack', type: Logger.ERROR); + }); + exitPort.listen((_) { + if (!exited.isCompleted) exited.complete(); + dispose(); + onExit(); + }); } - Future close() async { - _server.httpServer?.close(); + void dispose() { + ready.close(); + exitPort.close(); + errors.close(); } } + +/// What a server isolate boots from: the config, the command-line args and +/// the app's setup. +typedef _WorkerBoot = ( + Map, + List, + FutureOr Function()?, +); + +Future _workerMain((_WorkerBoot, SendPort) boot) async { + final ((config, args, setup), ready) = boot; + _isWorker = true; + + final app = Application(); + await app.initialize(config: config, args: args, setup: setup); + + final control = ReceivePort(); + ready.send(control.sendPort); + await control.first; + await app._server?.shutdown(timeout: shutdownTimeout); + await Logger.flush(); + Isolate.exit(); +} diff --git a/packages/core/lib/src/enum/http_request_method.dart b/packages/core/lib/src/enum/http_request_method.dart index 99e85ec..3b419b0 100644 --- a/packages/core/lib/src/enum/http_request_method.dart +++ b/packages/core/lib/src/enum/http_request_method.dart @@ -13,4 +13,7 @@ enum HttpRequestMethod { lock, unlock, propfind, + + /// A safe, idempotent query carried in the request content (RFC 10008). + query, } diff --git a/packages/core/lib/src/env_handler/env.dart b/packages/core/lib/src/env_handler/env.dart index de89e49..4c17cef 100644 --- a/packages/core/lib/src/env_handler/env.dart +++ b/packages/core/lib/src/env_handler/env.dart @@ -28,9 +28,17 @@ class Env { /// Evn.get('PORT', 3000); /// Evn.get('APP_KEY'); /// ``` + /// The types [get] parses a value into. + static const Set _typed = {'int', 'double', 'num', 'bool'}; + static T get(String key, [dynamic defaultValue]) { dynamic value = Env().env[key]; value ??= Platform.environment[key]; + // `KEY=` with nothing after it is a key left unset, not a number or a + // flag that failed to parse: a typed read falls back to the default. + if (value is String && value.trim().isEmpty && _typed.contains('$T')) { + value = null; + } value ??= defaultValue; if (T.toString() == 'int') { return int.parse(value.toString()) as T; diff --git a/packages/core/lib/src/exception/base_http_exception.dart b/packages/core/lib/src/exception/base_http_exception.dart index 25c3757..794b7d1 100644 --- a/packages/core/lib/src/exception/base_http_exception.dart +++ b/packages/core/lib/src/exception/base_http_exception.dart @@ -1,3 +1,5 @@ +import 'dart:convert' show htmlEscape; + import 'package:vania/src/http/response/response.dart'; class BaseHttpResponseException { @@ -12,6 +14,14 @@ class BaseHttpResponseException { }); Response response(bool isHtml) => isHtml - ? Response.html(message) + ? Response.html(htmlText, statusCode: code) : Response.json(message is Map ? message : {'message': message}, code); + + /// [message] as text for an HTML page, escaped: it may echo request + /// input. A map message shows its `message` entry. + String get htmlText { + final m = message; + final text = m is Map && m.containsKey('message') ? m['message'] : m; + return htmlEscape.convert(text.toString()); + } } diff --git a/packages/core/lib/src/exception/throttle_exception.dart b/packages/core/lib/src/exception/throttle_exception.dart index 91e36c2..7846ace 100644 --- a/packages/core/lib/src/exception/throttle_exception.dart +++ b/packages/core/lib/src/exception/throttle_exception.dart @@ -13,7 +13,11 @@ class ThrottleException extends BaseHttpResponseException { @override Response response(bool isHtml) { if (isHtml) { - return Response.html(message); + return Response.html( + htmlText, + statusCode: code, + headers: headers ?? const {}, + ); } final Map responseData = message is Map diff --git a/packages/core/lib/src/http/controller/controller_handler.dart b/packages/core/lib/src/http/controller/controller_handler.dart index 4aebf30..00e6760 100644 --- a/packages/core/lib/src/http/controller/controller_handler.dart +++ b/packages/core/lib/src/http/controller/controller_handler.dart @@ -1,17 +1,6 @@ -import 'dart:convert' show htmlEscape; - -import 'package:vania/env.dart' show env; -import 'package:vania/src/exception/database_exception.dart'; -import 'package:vania/src/logger/logger.dart'; -import 'package:vania/src/exception/query_exception.dart'; -import 'package:vania/src/exception/validation_exception.dart'; import 'package:vania/src/http/request/request.dart'; import 'package:vania/src/http/response/response.dart'; import 'package:vania/src/route/route_data.dart'; -import 'package:vania/src/route/route_history.dart'; - -import '../../exception/base_http_exception.dart'; -import '../../exception/invalid_argument_exception.dart'; class ControllerHandler { /// Converts a route parameter from its URL text to the value passed to @@ -67,97 +56,44 @@ class ControllerHandler { return _isPlainInteger(whole) || whole == '0' || whole == '-0'; } - Future create({ + /// Calls the route's action and returns its [Response], unwritten. + /// + /// Writing it is left to the request handler, once the middleware chain + /// has finished, so a middleware wrapping the chain can still add to + /// the response after `next()` returns. An error is not handled here + /// either: it travels back out through the middleware, which may want + /// to see it, to the one place that turns errors into responses. + Future create({ required RouteData route, required Request request, }) async { - List positionalArguments = []; - if (route.params != null) { - try { - positionalArguments = route.params!.values - .map((param) => _getParamValue(param.toString())) - .toList(); - } on FormatException catch (e, stack) { - return _fail(request, e, stack, 500); - } - } - - if (route.hasRequest) { - positionalArguments.insert(0, request); - } - - try { - Response response = await Function.apply( - route.action, - positionalArguments, - {}, + final positionalArguments = [if (route.hasRequest) request]; + final declared = route.paramTypes; + route.params?.forEach((name, value) { + // A `where*` type was already checked and applied when the route + // matched; only undeclared params are inferred from their text. + positionalArguments.add( + declared != null && declared.containsKey(name) + ? value + : _getParamValue(value.toString()), ); - - await response.makeResponse(request.response); - } on ValidationException catch (error) { - bool isHtml = request.request.headers - .value('accept') - .toString() - .contains('html'); - if (isHtml) { - await Response.redirect( - RouteHistory().previousRoute, - ).makeResponse(request.response); - } else { - await error.response(false).makeResponse(request.response); - } - } on BaseHttpResponseException catch (error, stack) { - // Exceptions the application raised deliberately (401, 403, 404, - // 419, 429 …). Their message is written for the client, so it is - // safe to pass through — but a 5xx raised this way is still an - // internal failure and gets the same treatment as any other. - if (error.code >= 500) { - await _fail(request, error, stack, error.code); - } else { - await _send(request, error.message, error.code); - } - } on InvalidArgumentException catch (error, stack) { - await _fail(request, error, stack, 500); - } on DatabaseException catch (error, stack) { - await _fail(request, error, stack, 500); - } on QueryException catch (error, stack) { - await _fail(request, error, stack, 500); - } catch (error, stack) { - await _fail(request, error, stack, 500); - } + }); + return await _invoke(route.action, positionalArguments); } } -/// Handles an internal failure: logs it with its stack trace, and only -/// reveals the underlying message when `APP_DEBUG` is on. +/// Calls [action] with [args]. /// -/// Driver errors routinely carry SQL text, table and column names, so -/// they are not returned to the client outside debug mode. -Future _fail( - Request req, - Object error, - StackTrace stack, - int statusCode, -) async { - Logger.log('$error\n$stack', type: Logger.ERROR); - final message = env('APP_DEBUG', false) - ? error.toString() - : 'Internal Server Error'; - await _send(req, message, statusCode); -} - -Future _send(Request req, String message, int statusCode) async { - if (req.headers['accept'].toString().contains('html')) { - // Escape before embedding in HTML. Error text routinely contains - // caller-supplied input (a bad route param, a rejected field value), - // so writing it raw turned any error path into reflected XSS. - await Response.html( - htmlEscape.convert(message), - statusCode: statusCode, - ).makeResponse(req.response); - } else { - await Response.json({ - "message": message, - }, statusCode).makeResponse(req.response); - } +/// A direct call for the arities routes use in practice: `Function.apply` +/// costs some thirty times as much and runs on every request. +dynamic _invoke(Function action, List args) { + final dynamic f = action; + return switch (args.length) { + 0 => f(), + 1 => f(args[0]), + 2 => f(args[0], args[1]), + 3 => f(args[0], args[1], args[2]), + 4 => f(args[0], args[1], args[2], args[3]), + _ => Function.apply(action, args), + }; } diff --git a/packages/core/lib/src/http/middleware/middleware_handler.dart b/packages/core/lib/src/http/middleware/middleware_handler.dart index 4301523..90148ca 100644 --- a/packages/core/lib/src/http/middleware/middleware_handler.dart +++ b/packages/core/lib/src/http/middleware/middleware_handler.dart @@ -8,21 +8,27 @@ import 'middleware.dart'; /// either side of it — or skip the remainder by not calling it. /// /// A middleware that throws aborts the chain; [terminal] does not run. +/// One that calls `next` a second time gets a [StateError]: running the +/// rest of the chain twice would run the controller twice. Future middlewareHandler( List middlewares, Request request, [ Future Function()? terminal, -]) async { - var index = 0; - - Future next() async { - if (index >= middlewares.length) { - if (terminal != null) await terminal(); - return; - } - final middleware = middlewares[index++]; - await middleware.process(request, next); +]) { + Future run(int index) { + if (index >= middlewares.length) return terminal?.call() ?? Future.value(); + final middleware = middlewares[index]; + var called = false; + return middleware.process(request, () { + if (called) { + throw StateError( + '${middleware.runtimeType} called next() more than once.', + ); + } + called = true; + return run(index + 1); + }); } - await next(); + return run(0); } diff --git a/packages/core/lib/src/http/request/accept_query.dart b/packages/core/lib/src/http/request/accept_query.dart new file mode 100644 index 0000000..f70a965 --- /dev/null +++ b/packages/core/lib/src/http/request/accept_query.dart @@ -0,0 +1,31 @@ +import 'dart:io'; + +/// What may appear unquoted in a Structured Field token (RFC 9651 §3.3.4). +final RegExp _sfToken = RegExp(r"^[A-Za-z*][!#$%&'*+\-.^_`|~0-9A-Za-z:/]*$"); + +/// [mediaTypes] as an `Accept-Query` value (RFC 10008 §3): a Structured +/// Field list in which a media type that is not a valid token — one +/// starting with a digit, say — is sent as a quoted string. +String acceptQueryHeader(List mediaTypes) => [ + for (final type in mediaTypes) + _sfToken.hasMatch(type) + ? type + : '"${type.replaceAll(r'\', r'\\').replaceAll('"', r'\"')}"', +].join(', '); + +/// Whether [contentType] is one of [accepted], which may hold `type/*` +/// and `*/*`. +bool acceptsMediaType(List accepted, ContentType contentType) { + final primary = contentType.primaryType.toLowerCase(); + final sub = contentType.subType.toLowerCase(); + for (final range in accepted) { + final slash = range.indexOf('/'); + if (slash < 0) continue; + final rangePrimary = range.substring(0, slash); + final rangeSub = range.substring(slash + 1); + if (rangePrimary == '*' && rangeSub == '*') return true; + if (rangePrimary != primary) continue; + if (rangeSub == '*' || rangeSub == sub) return true; + } + return false; +} diff --git a/packages/core/lib/src/http/request/request.dart b/packages/core/lib/src/http/request/request.dart index 4a6bbd4..d11d98a 100644 --- a/packages/core/lib/src/http/request/request.dart +++ b/packages/core/lib/src/http/request/request.dart @@ -1,10 +1,14 @@ +import 'dart:collection'; import 'dart:convert'; import 'package:vania/src/http/session/flash_messages.dart'; import 'dart:io'; +import 'dart:typed_data'; import 'package:vania/env.dart' show env; import 'package:vania/src/contract/http/request/form_validation.dart'; import 'package:vania/src/exception/validation_exception.dart'; +import 'package:vania/src/http/request/accept_query.dart'; import 'package:vania/src/http/request/request_body.dart'; +import 'package:vania/src/utils/helper.dart' show abort; import 'package:vania/src/http/request/request_file.dart'; import 'package:vania/src/http/validation/custom_validation_rule.dart'; import 'package:vania/src/http/validation/validation_chain/validation.dart'; @@ -30,6 +34,12 @@ void clearRequestUserResolver() { _requestUserResolver = null; } +/// Peers whose `X-Forwarded-For` is believed, from `TRUSTED_PROXIES`. +final List _trustedProxies = env( + 'TRUSTED_PROXIES', + '', +).split(',').map((e) => e.trim()).where((e) => e.isNotEmpty).toList(); + class Request { late HttpRequest request; RouteData? route; @@ -63,10 +73,7 @@ class Request { final peer = ip; if (peer == null) return null; - final trusted = env( - 'TRUSTED_PROXIES', - '', - ).split(',').map((e) => e.trim()).where((e) => e.isNotEmpty).toList(); + final trusted = _trustedProxies; if (trusted.isEmpty) return peer; if (!trusted.contains('*') && !trusted.contains(peer)) return peer; @@ -96,13 +103,13 @@ class Request { Map get _query => uri.queryParameters; Map body = {}; - final Map _cookies = {}; - Map? _allData; + /// Parsed on first use: most requests never read a cookie, and those + /// that do — middleware included — may do so before the body is read. + late final Map _cookies = _parseCookies(); - void _rebuildAll() { - _allData = {...body, ..._query, ...params()}; - } + /// The merged body, query and route params, built on first use. + Map? _allData; Map get _all => _allData ??= {...body, ..._query, ...params()}; @@ -125,14 +132,15 @@ class Request { }; } - /// Extracts the cookies from the headers and stores them in [_cookies]. + /// Parses the cookies from the headers. /// /// Malformed pairs (no `=`) are skipped. Values /// that contain `=` characters (e.g. base64 with `=` padding) are /// preserved by rejoining everything after the first `=`. - void _extractCookies() { + Map _parseCookies() { + final parsed = {}; final List? cookies = _httpHeaders[HttpHeaders.cookieHeader]; - if (cookies == null) return; + if (cookies == null) return parsed; for (final header in cookies) { for (final pair in header.split(';')) { final eq = pair.indexOf('='); @@ -140,9 +148,10 @@ class Request { final name = pair.substring(0, eq).trim(); final value = pair.substring(eq + 1).trim(); if (name.isEmpty) continue; - _cookies[name] = value; + parsed[name] = value; } } + return parsed; } bool _bodyExtracted = false; @@ -157,20 +166,54 @@ class Request { Future extractBody() async { if (_bodyExtracted) return this; _bodyExtracted = true; - _extractCookies(); final method = request.method.toLowerCase(); if (method == 'post' || method == 'patch' || method == 'put' || - method == 'delete') { - body = await RequestBody.extractBody(request: request); + method == 'delete' || + method == 'query') { + final isQuery = method == 'query'; + if (isQuery) _checkQueryMediaType(); + final content = await RequestBody.extract( + request: request, + strict: isQuery, + ); + body = content.fields; + _rawBody = content.bytes; + // Anything read before now saw no body; rebuild on next use. + _allData = null; } - _rebuildAll(); return this; } + Uint8List? _rawBody; + + /// The request content as received, before any parsing — for a media + /// type the framework does not read into fields, such as the + /// `application/sql` of a `QUERY`, or to verify a webhook signature. + /// + /// Empty before the body has been read, and for multipart uploads, + /// which are streamed rather than held. + Uint8List get rawBody => _rawBody ?? Uint8List(0); + + /// A `QUERY` is its content, so the server must know what that content + /// is: RFC 10008 has it fail a request without a `Content-Type`, and + /// cannot have it guess one. + void _checkQueryMediaType() { + final type = request.headers.contentType; + if (type == null) { + abort(400, 'A QUERY request must give its media type in Content-Type.'); + return; + } + final accepted = route?.acceptQuery; + if (accepted != null && !acceptsMediaType(accepted, type)) { + response.headers.set('Accept-Query', acceptQueryHeader(accepted)); + abort(415, 'This QUERY does not take ${type.mimeType} content.'); + } + } + Map all() { - return Map.unmodifiable(_all); + return UnmodifiableMapView(_all); } Map params() { @@ -191,11 +234,11 @@ class Request { bool has(dynamic keys) { if (keys is String) { - String? val = _all[keys]; + final val = _all[keys]; if (val == null) { return false; } - return val.toString().isNotEmpty ? true : false; + return val.toString().isNotEmpty; } if (keys is List) { @@ -230,7 +273,9 @@ class Request { } Map except(dynamic key) { - Map requestItems = _all; + // A copy: removing from the merged view itself would drop the keys + // from every later read of this request's input. + final requestItems = Map.of(_all); if (key is List) { for (String vKey in key) { @@ -265,33 +310,23 @@ class Request { return defaultVal; } - RequestFile? file(String key) { - if (_all[key] == null) { - return null; - } - - if (_all[key] is! RequestFile) { - return (_all[key] as List).first; - } - - return _all[key]; - } + /// The upload under [key], or the first of several. Null when [key] + /// holds no upload, a plain text field included. + RequestFile? file(String key) => files(key)?.first; bool hasFile(String key) => (_all[key].toString().isNotEmpty && (file(key) != null || files(key) != null)); + /// The uploads under [key]. Null when it holds none. List? files(String key) { - if (_all[key] == null) { - return null; - } - - var files = _all[key]; - - if (files is! List) { - return [files]; + final value = _all[key]; + if (value is RequestFile) return [value]; + if (value is List) { + final uploads = value.whereType().toList(); + if (uploads.isNotEmpty) return uploads; } - return files as List; + return null; } String string(String key) { @@ -375,7 +410,7 @@ class Request { Map get headers { Map ret = {}; _httpHeaders.forEach((String name, List values) { - ret[name] = values.join(); + ret[name] = values.join(', '); }); return ret; } diff --git a/packages/core/lib/src/http/request/request_body.dart b/packages/core/lib/src/http/request/request_body.dart index 10fdf63..59ff1ab 100644 --- a/packages/core/lib/src/http/request/request_body.dart +++ b/packages/core/lib/src/http/request/request_body.dart @@ -2,6 +2,7 @@ import 'dart:convert'; import 'dart:io'; import 'dart:typed_data'; +import 'package:vania/src/exception/http_exception.dart'; import 'package:vania/src/http/request/request_form_data.dart'; import 'package:vania/src/utils/helper.dart' show abort; import 'package:vania/env.dart' show env; @@ -18,6 +19,18 @@ class RequestBody { static Future> extractBody({ required HttpRequest request, + }) async => (await extract(request: request)).fields; + + /// Reads the request content: [RequestContent.fields] parsed from a + /// form or JSON body, and [RequestContent.bytes] as received. + /// + /// Content that does not parse as its declared type yields no fields. + /// With [strict] it is a 400 instead: RFC 10008 has a `QUERY` server + /// reject content that contradicts its `Content-Type`, since the query + /// is that content and there is nothing else to fall back on. + static Future extract({ + required HttpRequest request, + bool strict = false, }) async { final headers = request.headers; final contentLength = headers.contentLength; @@ -32,35 +45,57 @@ class RequestBody { final contentType = request.headers.contentType; if (isFormData(contentType)) { - final formData = RequestFormData(request: request); - await formData.extractData(); - return formData.inputs; + final formData = RequestFormData( + request: request, + maxBodySize: _maxBodySizeBytes, + ); + try { + await formData.extractData(); + } on HttpResponseException { + rethrow; + } catch (_) { + if (strict) abort(400, 'The content is not valid multipart data.'); + rethrow; + } + // Parts are streamed rather than buffered, so there are no bytes to + // keep. + return RequestContent(formData.inputs, Uint8List(0)); } final Uint8List bodyBytes = await _drain(request, contentLength); - if (bodyBytes.isEmpty) return {}; + if (bodyBytes.isEmpty) return RequestContent({}, bodyBytes); if (isJson(contentType)) { try { final decoded = _utf8Json.convert(bodyBytes); - if (decoded is Map) return decoded; + if (decoded is Map) { + return RequestContent(decoded, bodyBytes); + } if (decoded is Map) { - return decoded.cast(); + return RequestContent(decoded.cast(), bodyBytes); } - } catch (_) {} - return {}; + // Valid JSON that is not an object has no fields to offer. + return RequestContent({}, bodyBytes); + } catch (_) { + if (strict) abort(400, 'The content is not valid JSON.'); + return RequestContent({}, bodyBytes); + } } if (isUrlencoded(contentType)) { try { - return Uri.splitQueryString(utf8.decode(bodyBytes)); + return RequestContent( + Uri.splitQueryString(utf8.decode(bodyBytes)), + bodyBytes, + ); } catch (_) { - return {}; + if (strict) abort(400, 'The content is not valid form data.'); + return RequestContent({}, bodyBytes); } } - return {}; + return RequestContent({}, bodyBytes); } static Future _drain( @@ -146,3 +181,14 @@ class RequestBody { return mime.contains('json'); } } + +/// A request body as read by [RequestBody.extract]. +class RequestContent { + /// Fields parsed from a form or JSON object; empty for anything else. + final Map fields; + + /// The content as received. Empty for multipart, which is streamed. + final Uint8List bytes; + + const RequestContent(this.fields, this.bytes); +} diff --git a/packages/core/lib/src/http/request/request_file.dart b/packages/core/lib/src/http/request/request_file.dart index 89cb297..84f6ca0 100644 --- a/packages/core/lib/src/http/request/request_file.dart +++ b/packages/core/lib/src/http/request/request_file.dart @@ -30,7 +30,7 @@ class RequestFile { /// Lazily reads all bytes from the multipart stream. Future get bytes async { if (_bytes != null) return _bytes!; - final builder = BytesBuilder(); + final builder = BytesBuilder(copy: false); await for (final chunk in stream) { builder.add(chunk); } @@ -58,7 +58,7 @@ class RequestFile { Future store({String path = '', required String name}) async { try { final content = await bytes; - return await Storage.put(path, name, content.toList()); + return await Storage.put(path, name, content); } catch (e) { throw FileSystemException('Failed to store file as $name: $e'); } diff --git a/packages/core/lib/src/http/request/request_form_data.dart b/packages/core/lib/src/http/request/request_form_data.dart index c119aa3..0743f9e 100644 --- a/packages/core/lib/src/http/request/request_form_data.dart +++ b/packages/core/lib/src/http/request/request_form_data.dart @@ -1,9 +1,12 @@ +import 'dart:async'; import 'dart:convert'; import 'dart:io'; import 'dart:typed_data'; import 'package:mime/mime.dart'; +import 'package:vania/src/utils/helper.dart' show abort; + import 'request_file.dart'; final RegExp _dispositionParam = RegExp(r'([\w\-]+)="([^"]*)"'); @@ -14,7 +17,11 @@ class RequestFormData { final HttpRequest request; final Map inputs = {}; - RequestFormData({required this.request}); + /// Largest body accepted, counted as it streams in. A chunked upload + /// declares no length up front, so this is the only check it meets. + final int? maxBodySize; + + RequestFormData({required this.request, this.maxBodySize}); /// Parse and collect all form-data parts from the request. /// - For text fields: decodes UTF-8 and converts to int if possible. @@ -26,8 +33,45 @@ class RequestFormData { } final transformer = MimeMultipartTransformer(boundary); - final parts = request.cast>().transform(transformer); + final source = request.cast>(); + + final max = maxBodySize; + if (max == null) { + await _collect(source.transform(transformer)); + return this; + } + + // Reading stops at the first chunk past the cap. The parser cannot be + // relied on to finish after that: a part cut short is never closed, so + // anything still reading it would wait forever. The cut is raced + // against the parse instead. + final tooLarge = Completer(); + var received = 0; + final capped = source.takeWhile((chunk) { + received += chunk.length; + if (received <= max) return true; + tooLarge.complete(); + return false; + }); + + try { + await Future.any([ + _collect(capped.transform(transformer)), + tooLarge.future, + ]); + } catch (_) { + if (!tooLarge.isCompleted) rethrow; + } + if (tooLarge.isCompleted) { + abort( + 413, + 'Request body too large: streamed $received bytes (max $max bytes)', + ); + } + return this; + } + Future _collect(Stream parts) async { await for (final part in parts) { final disposition = part.headers['content-disposition']; if (disposition == null) continue; @@ -81,7 +125,5 @@ class RequestFormData { } } } - - return this; } } diff --git a/packages/core/lib/src/http/request/request_handler.dart b/packages/core/lib/src/http/request/request_handler.dart index 8d57ef0..1be9ae2 100644 --- a/packages/core/lib/src/http/request/request_handler.dart +++ b/packages/core/lib/src/http/request/request_handler.dart @@ -1,22 +1,24 @@ import 'dart:convert' show htmlEscape; +import 'dart:async'; import 'dart:io'; -import 'package:vania/src/exception/database_exception.dart'; -import 'package:vania/src/exception/query_exception.dart'; import 'package:vania/src/extensions/extensions.dart'; import 'package:vania/src/http/middleware/middleware.dart'; import 'package:vania/src/http/request/request_scope.dart'; +import 'package:vania/src/http/request/accept_query.dart'; +import 'package:vania/src/http/response/error_response_writer.dart'; import 'package:vania/src/http/response/response.dart'; import 'package:vania/src/http/session/session_gate.dart'; import 'package:vania/src/http/session/session_manager.dart'; import 'package:vania/src/route/middleware/csrf_middleware.dart'; -import 'package:vania/src/view_engine/helper.dart'; +import 'package:vania/src/view_engine/template_engine.dart'; import 'package:vania/src/config/http_cors.dart'; import 'package:vania/src/exception/internal_server_error.dart'; -import 'package:vania/src/exception/invalid_argument_exception.dart'; import 'package:vania/src/exception/page_expired_exception.dart'; import 'package:vania/src/exception/not_found_exception.dart'; import 'package:vania/src/exception/unauthenticated.dart'; +import 'package:vania/src/exception/validation_exception.dart'; +import 'package:vania/src/exception/redirect_exception.dart'; import 'package:vania/src/http/controller/controller_handler.dart'; import 'package:vania/src/http/middleware/middleware_handler.dart'; import 'package:vania/src/ioc_container.dart'; @@ -48,13 +50,33 @@ final bool _debug = () { } }(); +/// Longest a routed request may take to start answering, from +/// `REQUEST_TIMEOUT` in seconds; off when unset. Reading the request body +/// counts, so a slow upload can run out of it. A response already being +/// written — an event stream, a large download — runs to its end. +final Duration? _requestTimeout = () { + final seconds = env('REQUEST_TIMEOUT', 0); + return seconds > 0 ? Duration(seconds: seconds) : null; +}(); + /// A single CSRF middleware instance, attached to every state-changing /// route. `RouteData` is shared between requests, so per-request /// instances must never be appended to it. final CsrfMiddleware _csrfMiddleware = CsrfMiddleware(); class RequestHandler { + /// Handles [req] end to end. Never completes with an error: the server + /// does not await this future, so an error escaping it would be + /// unhandled and stop the isolate. Future handle(HttpRequest req) async { + try { + await _handle(req); + } catch (e, stack) { + await _fail(req, e, stack); + } + } + + Future _handle(HttpRequest req) async { if (_websocketEnabled && WebSocketTransformer.isUpgradeRequest(req)) { // A rejected or failed upgrade must not surface as an unhandled async // error: the socket is hijacked by then, so there is no response left @@ -71,7 +93,42 @@ class RequestHandler { if (await setStaticPathAsync(req)) return; - return runInRequestScope(req, () => _handleInScope(req)); + final timeout = _requestTimeout; + if (timeout == null) { + return runInRequestScope(req, () => _handleInScope(req)); + } + late final RequestScope scope; + final handled = runInRequestScope(req, () { + scope = currentRequestScope!; + return _handleInScope(req); + }); + // Set outside the request's zone, so the 503 it writes is not taken + // for a late write of the abandoned request. + final timer = Timer(timeout, () { + if (!scope.responding) unawaited(_timedOut(req, scope, timeout)); + }); + try { + await handled; + } finally { + timer.cancel(); + } + } + + /// Answers a request that ran past `REQUEST_TIMEOUT`. The handler itself + /// cannot be stopped and runs on, but its connection is released and + /// nothing it does from here reaches the client or the session. + Future _timedOut( + HttpRequest req, + RequestScope scope, + Duration timeout, + ) async { + scope.abandoned = true; + Logger.log( + 'Request timed out after ${timeout.inSeconds}s: ' + '${req.method} ${req.uri.path}', + type: Logger.WARNING, + ); + await _response(req, 'Request timed out', statusCode: 503); } Future _handleInScope(HttpRequest req) async { @@ -81,8 +138,17 @@ class RequestHandler { // one regardless of what the client asks for. final bool isHtml = acceptHeader.contains('html') && sessionsEnabled(); try { - final RouteData? route = httpRouteHandler(req); - if (route == null) return; + final RouteData? route = matchRoute(req); + if (route == null) return await _notFound(req, isHtml); + + // RFC 10008 §3: a QUERY resource says which media types it takes. + final acceptQuery = route.acceptQuery; + if (acceptQuery != null) { + req.response.headers.set( + 'Accept-Query', + acceptQueryHeader(acceptQuery), + ); + } final DateTime? startTime = _debug ? DateTime.now() : null; @@ -98,7 +164,10 @@ class RequestHandler { final List pre = _effectiveMiddleware(route); + Response? response; + var dispatched = false; Future dispatch() async { + dispatched = true; await request.extractBody(); if (isHtml) { @@ -108,7 +177,10 @@ class RequestHandler { } } - await ControllerHandler().create(route: route, request: request); + response = await ControllerHandler().create( + route: route, + request: request, + ); } if (pre.isEmpty) { @@ -117,6 +189,34 @@ class RequestHandler { await middlewareHandler(pre, request, dispatch); } + // A middleware may end the chain by answering itself. One that ends + // it without answering would leave the client waiting for good. + if (!dispatched && !_responseStarted(req.response)) { + throw StateError( + 'A middleware ended the chain without answering: it neither ' + 'called next() nor wrote a response. Middleware on this route: ' + '${pre.map((m) => m.runtimeType).join(', ')}.', + ); + } + + // Written only now, after every middleware has finished, so one that + // wraps the chain can still add to the response once next() returns. + // None, when a middleware answered the request itself. + if (response != null) { + currentRequestScope?.responding = true; + await response!.makeResponse(req.response); + } + + // Normally a no-op: `Response` already ran these before writing. A + // middleware that wrote the response itself skipped them, and a late + // session save still beats a lost one — unless the request was + // already answered without it. + final scope = currentRequestScope; + if (scope != null && !scope.abandoned) { + final pending = scope.runBeforeResponse(); + if (pending != null) await pending; + } + if (_debug && startTime != null) { final endTime = DateTime.now(); final duration = endTime.difference(startTime).inMilliseconds; @@ -129,45 +229,133 @@ class RequestHandler { '${startTime.format()} $requestMethod $requestedPath ~ ${duration}ms', ); } - } on BaseHttpResponseException catch (error) { - if (error is NotFoundException && isHtml) { - if (File('lib/resources/view/errors/404.html').existsSync()) { - return view('errors/404').makeResponse(req.response); - } + } catch (error, stack) { + if (currentRequestScope?.abandoned ?? false) { + // Answered already; the error is the handler's own, so it is + // still worth knowing about. + Logger.log('$error\n$stack', type: Logger.ERROR); + return; } + await _respondToError(req, error, stack, isHtml); + } + } - if (error is InternalServerError && isHtml) { - if (File('lib/resources/view/errors/500.html').existsSync()) { - return view('errors/500').makeResponse(req.response); - } - } + /// Turns an error from anywhere in a request — the router, a middleware, + /// the controller — into its response. One policy, so the same error is + /// answered the same way wherever it was raised. + Future _respondToError( + HttpRequest req, + Object error, + StackTrace stack, + bool isHtml, + ) async { + if (error is ValidationException) { + // The errors are flashed to the session, so a browser can be sent + // back to the form; anything else gets them here. + return sendErrorResponse( + req.response, + isHtml ? Response.back() : error.response(false), + ); + } - if (error is PageExpiredException && isHtml) { - if (File('lib/resources/view/errors/419.html').existsSync()) { - return view('errors/419').makeResponse(req.response); - } - } + if (error is! BaseHttpResponseException) { + return _fail(req, error, stack); + } - if (error is Unauthenticated && isHtml) { - return Response.redirect(error.message).makeResponse(req.response); + // Raised on purpose, yet a failure on the server all the same. + if (error.code >= 500) Logger.log('$error\n$stack', type: Logger.ERROR); + + if (isHtml) { + // A redirect has to be asked for: a session guard says so with + // `ResponseType.html` and the login page as its message. Anything + // else — a token guard's "Unauthenticated", say — is a 401, since a + // message is text, not an address to send the browser to. + final redirects = + error is RedirectException || + (error is Unauthenticated && error.responseType == ResponseType.html); + if (redirects) { + return sendErrorResponse( + req.response, + Response.redirect('${error.message}'), + ); } + final page = switch (error) { + NotFoundException() => 'errors/404', + InternalServerError() => 'errors/500', + PageExpiredException() => 'errors/419', + _ => null, + }; + final rendered = page == null ? null : _errorPage(page, error.code); + if (rendered != null) return sendErrorResponse(req.response, rendered); + } + + // A 5xx message may describe internals, so it is shown only in debug. + if (error.code >= 500) { + return _response( + req, + _publicMessage('${error.message}'), + statusCode: error.code, + ); + } + await sendErrorResponse(req.response, error.response(isHtml)); + } - if (error is RedirectException && isHtml) { - return Response.redirect(error.message).makeResponse(req.response); + /// Answers a request that matches no route. Unmatched paths are common, + /// so this is reached by value rather than by throwing. + Future _notFound(HttpRequest req, bool isHtml) async { + if (req.method.toUpperCase() == 'OPTIONS') { + // A preflight for a path with no route of its own; the CORS headers + // are already on the response. `Allow` says what the path takes — + // how a client finds out whether it supports QUERY. + final routes = routesForPath(req); + final headers = req.response.headers; + if (routes.isNotEmpty) { + headers.set( + HttpHeaders.allowHeader, + [ + for (final route in routes) ...[ + route.method.toUpperCase(), + if (route.method == 'get') 'HEAD', + ], + 'OPTIONS', + ].join(', '), + ); + } + for (final route in routes) { + final acceptQuery = route.acceptQuery; + if (acceptQuery != null) { + headers.set('Accept-Query', acceptQueryHeader(acceptQuery)); + } } + headers.contentLength = 0; + await req.response.close(); + return; + } + final rendered = isHtml ? _errorPage('errors/404', 404) : null; + if (rendered != null) return sendErrorResponse(req.response, rendered); + await routeNotFound().response(isHtml).makeResponse(req.response); + } - await error.response(isHtml).makeResponse(req.response); - } on InvalidArgumentException catch (e, stack) { - await _fail(req, e, stack); - } on DatabaseException catch (error, stack) { - await _fail(req, error, stack); - } on QueryException catch (error, stack) { - await _fail(req, error, stack); - } catch (e, stack) { - await _fail(req, e, stack); + /// Whether [res] has gone out, at least its headers. dart:io has no + /// direct way to ask; once the headers are sent, the status can no + /// longer be set. + static bool _responseStarted(HttpResponse res) { + try { + res.statusCode = res.statusCode; + return false; + } on StateError { + return true; } } + /// The app's own page for an error, such as `errors/404`, sent with the + /// error's [statusCode] — a 404 page answered with 200 tells crawlers and + /// monitoring that all is well. Null when the app has no such page. + Response? _errorPage(String page, int statusCode) { + if (!File('lib/resources/view/$page.html').existsSync()) return null; + return Response.html(TemplateEngine().render(page), statusCode: statusCode); + } + /// Returns the effective pre-middleware list for [route] without /// mutating the cached RouteData. If CSRF is disabled or the route /// already has no middleware, the caller receives the route's own @@ -175,13 +363,9 @@ class RequestHandler { List _effectiveMiddleware(RouteData route) { if (!_csrfEnabled) return route.preMiddleware; - final method = route.method; // CSRF only applies to state-changing methods. For everything else // the route's own list is fine. - if (method != 'post' && - method != 'put' && - method != 'patch' && - method != 'delete') { + if (!CsrfMiddleware.protectedMethods.contains(route.method)) { return route.preMiddleware; } @@ -193,34 +377,36 @@ class RequestHandler { } } - return [...route.preMiddleware, _csrfMiddleware]; + // First, so a forged request is turned away before any of the app's + // own middleware acts on it. + return [_csrfMiddleware, ...route.preMiddleware]; } /// Logs an internal failure with its stack trace and replies without - /// leaking the underlying message unless APP_DEBUG is on. Mirrors - /// `_fail` in ControllerHandler — same reasoning, different layer. + /// leaking the underlying message unless APP_DEBUG is on. Future _fail(HttpRequest req, Object error, StackTrace stack) async { Logger.log('$error\n$stack', type: Logger.ERROR); - final message = env('APP_DEBUG', false) - ? error.toString() - : 'Internal Server Error'; - await _response(req, message); + await _response(req, _publicMessage('$error')); } + /// [detail] in debug mode, and a generic message otherwise: errors from + /// drivers and the like routinely carry SQL, paths and other internals. + String _publicMessage(String detail) => + env('APP_DEBUG', false) ? detail : 'Internal Server Error'; + Future _response( HttpRequest req, String message, { int statusCode = 500, }) async { - if (req.headers.value('accept').toString().contains('html')) { - await Response.html( - htmlEscape.convert(message), - statusCode: statusCode, - ).makeResponse(req.response); - } else { - await Response.json({ - "message": message, - }, statusCode).makeResponse(req.response); - } + // HTML on the same terms as every other answer: the client asks for it + // and the app renders pages at all. An API answers in JSON. + final html = + (req.headers.value('accept') ?? '').contains('html') && + sessionsEnabled(); + final response = html + ? Response.html(htmlEscape.convert(message), statusCode: statusCode) + : Response.json({"message": message}, statusCode); + await sendErrorResponse(req.response, response); } } diff --git a/packages/core/lib/src/http/request/request_scope.dart b/packages/core/lib/src/http/request/request_scope.dart index b81b87b..4de38d8 100644 --- a/packages/core/lib/src/http/request/request_scope.dart +++ b/packages/core/lib/src/http/request/request_scope.dart @@ -35,7 +35,55 @@ class RequestScope { /// core having to know about them. final Map attributes = {}; + /// Whether [sessionData] has been read from the session store. + bool sessionLoaded = false; + + /// Set when the handler starts writing its response. From then on + /// `REQUEST_TIMEOUT` leaves the request alone: it limits how long the + /// app takes to answer, not how long a stream or a download runs. + bool responding = false; + + /// Set once the request has been answered without its handler — by + /// `REQUEST_TIMEOUT`. The handler cannot be stopped, but whatever it + /// goes on to write or save is dropped: the client has its answer. + bool abandoned = false; + + List Function()>? _beforeResponse; + RequestScope(this.request); + + /// Registers [task] to finish before this request's response is written. + /// + /// For writes the next request from the same client must see — a + /// session save, above all: once the response is out, the browser may + /// already be following its redirect. + void beforeResponse(Future Function() task) { + (_beforeResponse ??= Function()>[]).add(task); + } + + /// Runs the tasks registered with [beforeResponse], each once. + /// + /// `Response.makeResponse` calls this before writing; the request + /// handler calls it again when the request ends, for a response written + /// some other way. + /// + /// Returns null when there is nothing to run, so the common case costs + /// the caller no `await`. + Future? runBeforeResponse() { + if (_beforeResponse == null) return null; + return _runAll(); + } + + Future _runAll() async { + // A task may register another: saving flashed messages writes to the + // session, whose own save then has to follow. + for (var tasks = _beforeResponse; tasks != null; tasks = _beforeResponse) { + _beforeResponse = null; + for (final task in tasks) { + await task(); + } + } + } } const Object _requestScopeKey = #vania.request_scope; diff --git a/packages/core/lib/src/http/response/error_response_writer.dart b/packages/core/lib/src/http/response/error_response_writer.dart new file mode 100644 index 0000000..64d1bf3 --- /dev/null +++ b/packages/core/lib/src/http/response/error_response_writer.dart @@ -0,0 +1,25 @@ +import 'dart:io'; + +import 'response.dart'; + +/// Writes [response] on an error path. +/// +/// An error can surface after the response has already started: headers +/// sent, part of the body flushed, or the client gone. Nothing more can be +/// said to the client then, so the response is closed as it stands rather +/// than letting a second failure escape the request pipeline. +Future sendErrorResponse(HttpResponse res, Response response) async { + try { + await response.makeResponse(res); + } catch (_) { + await closeQuietly(res); + } +} + +/// Closes [res], ignoring a response that is already closed or whose +/// connection has dropped. +Future closeQuietly(HttpResponse res) async { + try { + await res.close(); + } catch (_) {} +} diff --git a/packages/core/lib/src/http/response/response.dart b/packages/core/lib/src/http/response/response.dart index b65d660..e1a2072 100644 --- a/packages/core/lib/src/http/response/response.dart +++ b/packages/core/lib/src/http/response/response.dart @@ -1,13 +1,23 @@ +import 'dart:async'; import 'dart:convert'; import 'package:vania/src/http/session/flash_messages.dart'; import 'dart:io'; import 'dart:typed_data'; import 'package:meta/meta.dart'; +import 'package:mime/mime.dart'; +import 'package:vania/env.dart' show env; +import 'package:vania/src/http/request/request_scope.dart'; +import 'package:vania/src/http/response/error_response_writer.dart'; import 'package:vania/src/http/response/stream_file.dart'; +import 'package:vania/src/logger/logger.dart'; import 'package:vania/src/route/route_history.dart'; final Utf8Encoder _utf8 = utf8.encoder; +/// How often an event stream sends a comment line while it has nothing +/// else to send, from `SSE_HEARTBEAT` in seconds. +final Duration _sseHeartbeat = Duration(seconds: env('SSE_HEARTBEAT', 15)); + /// Encodes straight from the object graph to UTF-8 bytes. /// /// `utf8.encode(jsonEncode(data))` builds a full intermediate `String` and @@ -38,21 +48,65 @@ class Response { @protected Future sseHandler(HttpResponse res) async { + // Buffered, the response holds chunks back even through `flush()`, and + // an event would only reach the client once 8 KB had piled up. + res.bufferOutput = false; res.headers.contentType = ContentType.parse('text/event-stream'); res.headers.add(HttpHeaders.cacheControlHeader, 'no-cache'); res.headers.add(HttpHeaders.connectionHeader, 'keep-alive'); res.headers.add(HttpHeaders.transferEncodingHeader, 'chunked'); - void writeSSE(String data) { - res.add(utf8.encode('data: $data\n\n')); + final finished = Completer(); + void finish() { + if (!finished.isCompleted) finished.complete(); } - await for (var event in data) { - writeSSE(jsonEncode(event)); - await res.flush(); + // One write at a time: the response refuses a write while a flush is + // in progress. A failed write means the client has gone — and so does + // one that does not finish: once the client has left, dart:io neither + // fails the flush nor completes `done`, it just never returns. + var writes = Future.value(); + void send(String text) { + writes = writes.then((_) { + res.add(utf8.encode(text)); + return res.flush().timeout(_sseHeartbeat); + }); + writes.catchError((Object _) => finish()); } - await res.close(); + final events = (data as Stream).listen(null); + events + ..onData((event) { + try { + send('data: ${jsonEncode(event)}\n\n'); + } catch (e, stack) { + Logger.log('SSE event not sent: $e\n$stack', type: Logger.ERROR); + finish(); + return; + } + // Holds the source back while a slow client catches up. + events.pause(writes.catchError((Object _) {})); + }) + ..onError((Object e, StackTrace stack) { + Logger.log('SSE event source failed: $e\n$stack', type: Logger.ERROR); + finish(); + }) + ..onDone(finish); + + // Keeps proxies from closing a quiet connection, and finds out when + // the client has left even if no event is due. + final heartbeat = Timer.periodic(_sseHeartbeat, (_) => send(':\n\n')); + res.done.then((_) => finish(), onError: (Object _) => finish()); + + await finished.future; + heartbeat.cancel(); + await events.cancel(); + // Bounded for the same reason: a response to a client that has left + // may never finish closing. + await writes + .catchError((Object _) {}) + .then((_) => closeQuietly(res)) + .timeout(_sseHeartbeat, onTimeout: () {}); } /// Writes the response to the wire and closes the underlying [HttpResponse]. @@ -60,6 +114,11 @@ class Response { /// Callers must await this: the returned future completes once the /// response has been flushed and the connection closed. Future makeResponse(HttpResponse res) async { + // Answered already, by REQUEST_TIMEOUT: nothing more goes out. + final scope = currentRequestScope; + if (scope != null && scope.abandoned) return; + final pending = scope?.runBeforeResponse(); + if (pending != null) await pending; res.statusCode = httpStatusCode; if (headers.isNotEmpty) { headers.forEach((key, value) { @@ -69,13 +128,21 @@ class Response { switch (responseType) { case ResponseType.json: res.headers.contentType = ContentType.json; + List bytes; try { - final bytes = _jsonUtf8.convert(data); - res.headers.contentLength = bytes.length; - res.add(bytes); - } catch (e) { - res.write('jsonEncode Error: $e'); + bytes = _jsonUtf8.convert(data); + } catch (e, stack) { + // Nothing is written yet, so the status can still say it failed. + Logger.log('$e\n$stack', type: Logger.ERROR); + res.statusCode = HttpStatus.internalServerError; + bytes = _jsonUtf8.convert({ + 'message': env('APP_DEBUG', false) + ? e.toString() + : 'Internal Server Error', + }); } + res.headers.contentLength = bytes.length; + res.add(bytes); await res.close(); break; case ResponseType.html: @@ -105,7 +172,7 @@ class Response { } res.headers.contentType = stream.contentType; res.headers.contentLength = stream.length; - await res.addStream(stream.stream!); + res.add(stream.bytes); await res.close(); break; case ResponseType.download: @@ -122,7 +189,7 @@ class Response { res.headers.contentType = stream.contentType; res.headers.contentLength = stream.length; res.headers.add("Content-Disposition", stream.contentDisposition); - await res.addStream(stream.stream!); + res.add(stream.bytes); await res.close(); break; case ResponseType.redirect: @@ -130,7 +197,7 @@ class Response { await res.close(); break; case ResponseType.none: - res.write(data); + if (data != null) res.write(data); await res.close(); break; } @@ -182,6 +249,27 @@ class Response { headers: headers, ); + /// Streams the file at [path] from disk without holding it in memory. + /// + /// A single `Range` request is answered with `206 Partial Content`, so + /// media can be seeked and interrupted downloads resumed. With + /// [download] the browser saves the file, as [fileName] or by default + /// under its own name. A path that is not a file gets a 404. + /// + /// [path] is read as given: never build it from request input without + /// confining it to a directory you mean to serve. + static Response fromPath( + String path, { + bool download = false, + String? fileName, + Map headers = const {}, + }) => _FileResponse( + path, + download: download, + fileName: fileName, + headers: headers, + ); + static Response sse( Stream eventStream, { int statusCode = HttpStatus.ok, @@ -204,40 +292,166 @@ class Response { ); static Response back([String? key, String? message]) { - String previousRoute = RouteHistory().previousRoute; if (key != null && message != null) { FlashMessages().flash(key, message); } - if (previousRoute.isNotEmpty) { - return Response( - responseType: ResponseType.redirect, - data: previousRoute, - httpStatusCode: HttpStatus.found, - ); - } - return Response( - responseType: ResponseType.redirect, - data: RouteHistory().currentRoute, - httpStatusCode: HttpStatus.found, - ); + return Response.redirect(_backUrl()); } static Response backWithInput([String? input, String? message]) { - String previousRoute = RouteHistory().previousRoute; if (input != null && message != null) { FlashMessages().addError(input, message); } - if (previousRoute.isNotEmpty) { - return Response( - responseType: ResponseType.redirect, - data: previousRoute, - httpStatusCode: HttpStatus.found, + return Response.redirect(_backUrl()); + } +} + +/// Where `back()` sends the client: the page it came from, as far as that +/// can be told safely, and `/` when it cannot. +/// +/// The `Referer` header is used only when it names this same site, and +/// only its path and query, so a crafted link cannot turn `back()` into a +/// redirect to somewhere else. +String _backUrl() { + final history = RouteHistory(); + if (history.previousRoute.isNotEmpty) return history.previousRoute; + + final req = currentRequestScope?.request; + final host = req?.headers.value(HttpHeaders.hostHeader)?.toLowerCase(); + final referer = Uri.tryParse( + req?.headers.value(HttpHeaders.refererHeader) ?? '', + ); + if (host != null && referer != null && referer.hasAuthority) { + final authority = referer.hasPort + ? '${referer.host}:${referer.port}' + : referer.host; + if (authority.toLowerCase() == host) { + // `//other.site` would be read as an address on another host. + final path = '/${referer.path.replaceFirst(RegExp('^/+'), '')}'; + return referer.hasQuery ? '$path?${referer.query}' : path; + } + } + + // The current path is a page to go back to only from a form post: a GET + // sent back to its own URL would answer itself with a redirect forever. + final method = req?.method.toUpperCase(); + if (history.currentRoute.isNotEmpty && method != 'GET' && method != 'HEAD') { + return history.currentRoute; + } + return '/'; +} + +/// The answer to a `Range` header no byte of the file satisfies. +const (int, int) _unsatisfiable = (-1, -1); + +class _FileResponse extends Response { + final String path; + final bool download; + final String? fileName; + + _FileResponse( + this.path, { + required this.download, + required this.fileName, + super.headers, + }); + + @override + Future makeResponse(HttpResponse res) async { + // Answered already, by REQUEST_TIMEOUT: nothing more goes out. + final scope = currentRequestScope; + if (scope != null && scope.abandoned) return; + final pending = scope?.runBeforeResponse(); + if (pending != null) await pending; + headers.forEach(res.headers.set); + + final file = File(path); + final stat = await file.stat(); + if (stat.type != FileSystemEntityType.file) { + return Response.json({ + 'message': 'File not found', + }, HttpStatus.notFound).makeResponse(res); + } + + final size = stat.size; + final lastModified = HttpDate.format(stat.modified.toUtc()); + res.headers + ..contentType = ContentType.parse( + lookupMimeType(path) ?? 'application/octet-stream', + ) + ..set(HttpHeaders.acceptRangesHeader, 'bytes') + ..set(HttpHeaders.lastModifiedHeader, lastModified); + if (download) { + res.headers.set( + 'Content-Disposition', + attachmentDisposition(fileName ?? path), ); } - return Response( - responseType: ResponseType.redirect, - data: RouteHistory().currentRoute, - httpStatusCode: HttpStatus.found, + + final range = _requestedRange( + currentRequestScope?.request, + size, + lastModified, ); + if (range == null) { + res.headers.contentLength = size; + // A HEAD gets the headers alone, so there is no reason to read the + // file. + if (currentRequestScope?.request.method.toUpperCase() != 'HEAD') { + await res.addStream(file.openRead()); + } + } else if (range == _unsatisfiable) { + res + ..statusCode = HttpStatus.requestedRangeNotSatisfiable + ..headers.set(HttpHeaders.contentRangeHeader, 'bytes */$size') + ..headers.contentLength = 0; + } else { + final (start, end) = range; + res + ..statusCode = HttpStatus.partialContent + ..headers.set(HttpHeaders.contentRangeHeader, 'bytes $start-$end/$size') + ..headers.contentLength = end - start + 1; + await res.addStream(file.openRead(start, end + 1)); + } + await res.close(); + } +} + +/// The inclusive byte range [req] asks for in a file of [size] bytes: +/// null to send the whole file, or [_unsatisfiable]. +/// +/// Only a single range is honoured. Several ranges would need a multipart +/// body; the whole file is a valid answer to that request too. +(int, int)? _requestedRange(HttpRequest? req, int size, String lastModified) { + final header = req?.headers.value(HttpHeaders.rangeHeader); + // RFC 10008 §2.8: ranges mean the same for QUERY as for GET. + final method = req?.method.toUpperCase(); + if (header == null || (method != 'GET' && method != 'QUERY')) return null; + + // `If-Range` asks for the range only if the file is still the version + // the client has part of; otherwise it wants the new one, whole. + final ifRange = req!.headers.value(HttpHeaders.ifRangeHeader); + if (ifRange != null && ifRange != lastModified) return null; + + if (!header.startsWith('bytes=')) return null; + final spec = header.substring(6).trim(); + if (spec.contains(',')) return null; + final dash = spec.indexOf('-'); + if (dash < 0) return null; + final first = spec.substring(0, dash).trim(); + final last = spec.substring(dash + 1).trim(); + + if (first.isEmpty) { + // `bytes=-N`: the last N bytes. + final suffix = int.tryParse(last); + if (suffix == null) return null; + if (suffix == 0 || size == 0) return _unsatisfiable; + return (suffix >= size ? 0 : size - suffix, size - 1); } + + final start = int.tryParse(first); + final end = last.isEmpty ? size - 1 : int.tryParse(last); + if (start == null || end == null || end < start) return null; + if (start >= size) return _unsatisfiable; + return (start, end < size ? end : size - 1); } diff --git a/packages/core/lib/src/http/response/stream_file.dart b/packages/core/lib/src/http/response/stream_file.dart index a44d67a..03e941c 100644 --- a/packages/core/lib/src/http/response/stream_file.dart +++ b/packages/core/lib/src/http/response/stream_file.dart @@ -1,3 +1,4 @@ +import 'dart:convert'; import 'dart:io'; import 'dart:typed_data'; import 'package:path/path.dart' as path; @@ -18,8 +19,7 @@ class StreamFile { Stream>? get stream => _stream; int get length => _length; - String get contentDisposition => - 'attachment; filename="${path.basename(fileName)}"'; + String get contentDisposition => attachmentDisposition(fileName); StreamFile? call() { String mimeType = @@ -37,3 +37,32 @@ class StreamFile { return this; } } + +/// Bytes RFC 8187 lets through `filename*` unencoded: `attr-char`. +final RegExp _attrChar = RegExp(r"[A-Za-z0-9!#$&+\-.^_`|~]"); + +/// A `Content-Disposition: attachment` value for [fileName] that any name +/// survives. +/// +/// HTTP header values are Latin-1, so a name outside printable ASCII — a +/// Persian one, say — cannot go in `filename` as it is. It goes in +/// `filename*` as UTF-8 (RFC 6266, RFC 8187), which browsers prefer, and +/// `filename` gets an ASCII stand-in for clients that do not read it. +/// Quotes, backslashes and control characters never reach the header, so +/// a name cannot end the value early or add a header of its own. +String attachmentDisposition(String fileName) { + final name = path.basename(fileName); + final ascii = name.replaceAll(RegExp(r'[^\x20-\x7E]|["\\]'), '_'); + if (ascii == name) return 'attachment; filename="$name"'; + + final encoded = StringBuffer(); + for (final byte in utf8.encode(name)) { + final char = String.fromCharCode(byte); + encoded.write( + byte < 0x80 && _attrChar.hasMatch(char) + ? char + : '%${byte.toRadixString(16).toUpperCase().padLeft(2, '0')}', + ); + } + return "attachment; filename=\"$ascii\"; filename*=UTF-8''$encoded"; +} diff --git a/packages/core/lib/src/http/session/flash_messages.dart b/packages/core/lib/src/http/session/flash_messages.dart index e63bd26..68077de 100644 --- a/packages/core/lib/src/http/session/flash_messages.dart +++ b/packages/core/lib/src/http/session/flash_messages.dart @@ -2,6 +2,7 @@ import 'dart:async'; import 'package:vania/src/ioc_container.dart'; import '../request/request_scope.dart'; +import 'session_manager.dart'; class FlashMessages { FlashMessages.createDefault(); @@ -33,26 +34,84 @@ class FlashMessages { Map get messages => currentRequestScope?.flashSessions ?? _state.messages; - /// Stores a flash message under [key]. - void flash(String key, dynamic value) => messages[key] = value; + /// The session entry flashed values travel to the next request in. + static const String sessionKey = '_flash'; + + /// Stores a flash message under [key], for this request and the next. + void flash(String key, dynamic value) { + messages[key] = value; + _outgoing()?.messages[key] = value; + } /// Reads a flash message, or null when absent. dynamic message(String key) => messages[key]; - /// Records a validation error for [field]. - void addError(String field, dynamic message) => errors[field] = message; + /// Records a validation error for [field], for this request and the + /// next. + void addError(String field, dynamic message) { + errors[field] = message; + _outgoing()?.errors[field] = message; + } - /// Merges [entries] into the current errors. - void addErrors(Map entries) => errors.addAll(entries); + /// Merges [entries] into the current errors, for this request and the + /// next. + void addErrors(Map entries) { + errors.addAll(entries); + _outgoing()?.errors.addAll(entries); + } /// Removes all validation errors for the current request. - void clearErrors() => errors.clear(); + void clearErrors() { + errors.clear(); + _existingOutgoing()?.errors.clear(); + } /// Removes every flash message, error and old-input entry. + /// + /// Rendering a view calls this once the page has used them, so nothing + /// it has shown is carried over to the next request. void clear() { errors.clear(); old.clear(); messages.clear(); + _existingOutgoing()?.clear(); + } + + /// What this request has flashed so far, registering its save on first + /// use. Null outside a request. + _Outgoing? _outgoing() { + final scope = currentRequestScope; + if (scope == null) return null; + final existing = _existingOutgoing(); + if (existing != null) return existing; + final created = _Outgoing(); + scope.attributes[_Outgoing] = created; + scope.beforeResponse(() => created.save(scope)); + return created; + } + + _Outgoing? _existingOutgoing() { + final value = currentRequestScope?.attributes[_Outgoing]; + return value is _Outgoing ? value : null; + } + + /// Hands this request what the previous one flashed, and removes it from + /// the session: flashed values are shown once. + static Future restore( + RequestScope scope, + SessionManager sessions, + ) async { + final flashed = scope.sessionData[sessionKey]; + if (flashed is! Map) return; + for (final (key, bucket) in [ + ('messages', scope.flashSessions), + ('errors', scope.sessionErrors), + ('old', scope.formData), + ]) { + final values = flashed[key]; + if (values is Map) bucket.addAll(values.cast()); + } + await sessions.deleteSession(sessionKey); } /// Runs [body] with its own isolated set of buckets, optionally @@ -77,3 +136,45 @@ class _FlashState { final Map formData = {}; final Map messages = {}; } + +/// What one request flashes for the next. +class _Outgoing { + final Map messages = {}; + final Map errors = {}; + + void clear() { + messages.clear(); + errors.clear(); + } + + Future save(RequestScope scope) async { + if (messages.isEmpty && errors.isEmpty) return; + await IoCContainer() + .resolveOrDefault(SessionManager.new) + .setSession(FlashMessages.sessionKey, { + if (messages.isNotEmpty) 'messages': messages, + if (errors.isNotEmpty) ...{ + 'errors': errors, + // The form comes back filled in, as submitted. + 'old': _storable(scope.formData), + }, + }); + } + + /// [input] as it may be kept in the session: no passwords, and nothing + /// that is not plain data, such as an upload. + static Map _storable(Map input) => { + for (final entry in input.entries) + if (!entry.key.toLowerCase().contains('password') && + _isPlain(entry.value)) + entry.key: entry.value, + }; + + static bool _isPlain(Object? value) => + value == null || + value is String || + value is num || + value is bool || + (value is List && value.every(_isPlain)) || + (value is Map && value.values.every(_isPlain)); +} diff --git a/packages/core/lib/src/http/session/session_file_store.dart b/packages/core/lib/src/http/session/session_file_store.dart index 9349301..138eed3 100644 --- a/packages/core/lib/src/http/session/session_file_store.dart +++ b/packages/core/lib/src/http/session/session_file_store.dart @@ -1,10 +1,19 @@ +import 'dart:async'; import 'dart:convert'; import 'dart:io'; +import 'dart:math'; import 'package:crypto/crypto.dart'; import '../../cryptographic/vania_encryption.dart'; import 'package:vania/env.dart' show env; +import 'package:vania/src/server/server_isolates.dart'; + +/// How long a session lasts after its last write, from `SESSION_LIFETIME` +/// in seconds. The cookie that carries it lasts as long. +final Duration sessionLifetime = Duration( + seconds: env('SESSION_LIFETIME', 9000), +); /// One entry in the in-memory LRU cache. class _CacheEntry { @@ -30,10 +39,18 @@ class SessionFileStore { /// LRU: insertion order is refresh order. On get, remove-and-reinsert to /// move the entry to the tail (most-recently-used). final Map _cache = {}; - int get _maxCacheSize => env('SESSION_CACHE_SIZE', 5000); + + /// Off when several isolates serve requests: each would hold its own + /// copy, and one isolate would serve a session another has changed. + int get _maxCacheSize => + serverIsolateCount > 1 ? 0 : env('SESSION_CACHE_SIZE', 5000); int get cacheSize => _cache.length; + final Random _random = Random(); + + String _tempSuffix() => _random.nextInt(1 << 32).toRadixString(36); + String filePathFor(String sessionId) => '$sessionPath/${_makeHash(sessionId)}'; @@ -68,14 +85,12 @@ class SessionFileStore { Future storeSession( String sessionId, Map data, { - Duration duration = const Duration(hours: 1), + Duration? duration, }) async { + _maybeSweep(); final hashedId = _makeHash(sessionId).toString(); final file = File('$sessionPath/$hashedId'); - if (!await file.exists()) { - await file.create(recursive: true); - } - final expiresAt = DateTime.now().toUtc().add(duration); + final expiresAt = DateTime.now().toUtc().add(duration ?? sessionLifetime); final sessionData = { 'data': data, 'expiration': expiresAt.millisecondsSinceEpoch, @@ -85,85 +100,121 @@ class SessionFileStore { _secretKey, ); - final raf = await file.open(mode: FileMode.write); + // Written beside the session, then renamed over it. A rename is + // atomic, so a request reading the session meanwhile finds the old one + // or the new one — never a file cut off halfway, which would read as + // no session at all and sign the user out. + final temp = File('${file.path}.tmp-${_tempSuffix()}'); + await temp.parent.create(recursive: true); + await temp.writeAsString(content); try { - await raf.writeFrom(utf8.encode(content)); - } finally { + await temp.rename(file.path); + } catch (_) { try { - await raf.unlock(); + await temp.delete(); } catch (_) {} - await raf.close(); + rethrow; } _put(hashedId, _CacheEntry(Map.of(data), expiresAt)); } /// Retrieves the session data, preferring the in-memory cache. - Future?> retrieveSession(String sessionId) async { + Future?> retrieveSession(String sessionId) async => + (await retrieveSessionEntry(sessionId))?.data; + + /// The session's data and when it expires, or null when there is none. + Future<({Map data, DateTime expiresAt})?> + retrieveSessionEntry(String sessionId) async { final hashedId = _makeHash(sessionId).toString(); final cached = _get(hashedId); - if (cached != null) return Map.of(cached.data); + if (cached != null) { + return (data: Map.of(cached.data), expiresAt: cached.expiresAt); + } final file = File('$sessionPath/$hashedId'); - if (!await file.exists()) return null; - final raf = await file.open(mode: FileMode.read); - String fileContent = ''; + final Map blob; try { - final length = await raf.length(); - fileContent = utf8.decode(await raf.read(length)); - } finally { - try { - await raf.unlock(); - } catch (_) {} - await raf.close(); + final decrypted = await VaniaEncryption.decryptString( + await file.readAsString(), + _secretKey, + ); + if (decrypted.isEmpty) return null; + blob = json.decode(decrypted) as Map; + } catch (_) { + // Missing, or unreadable — damaged, or written under another APP_KEY. + // No session, rather than an error on every request with this cookie. + return null; } - - final decrypted = await VaniaEncryption.decryptString( - fileContent, - _secretKey, - ); - if (decrypted.isEmpty) return null; - final blob = json.decode(decrypted) as Map; final expiration = int.tryParse(blob['expiration'].toString()) ?? 0; final expiresAt = DateTime.fromMillisecondsSinceEpoch(expiration).toUtc(); if (!DateTime.now().toUtc().isBefore(expiresAt)) { - await file.delete(); + try { + await file.delete(); + } catch (_) {} _cache.remove(hashedId); return null; } final data = Map.from(blob['data'] as Map); _put(hashedId, _CacheEntry(Map.of(data), expiresAt)); - return data; + return (data: data, expiresAt: expiresAt); + } + + /// When the last sweep for expired sessions ran in this isolate. + DateTime? _lastSweep; + + /// Deletes session files past their lifetime, and temporary files a + /// write left behind. + /// + /// An expired session is otherwise only deleted when someone reads it, + /// and most never come back: every visitor without a cookie leaves one. + /// A file's age is read from its modified time, which is when it was + /// last written — and every write sets a fresh expiry. + Future sweepExpired() async { + _lastSweep = DateTime.now(); + final now = DateTime.now(); + try { + await for (final entry in Directory(sessionPath).list()) { + if (entry is! File) continue; + try { + final age = now.difference(await entry.lastModified()); + final stale = entry.path.contains('.tmp-') + ? age > const Duration(minutes: 1) + : age > sessionLifetime; + if (stale) await entry.delete(); + } catch (_) { + // Gone already, or busy: the next sweep gets it. + } + } + } catch (_) { + // No session directory yet. + } + } + + void _maybeSweep() { + final last = _lastSweep; + if (last != null && + DateTime.now().difference(last) < const Duration(hours: 1)) { + return; + } + unawaited(sweepExpired()); } Future hasSession(String sessionId) async { return await retrieveSession(sessionId) != null; } - /// Marks the session as deleted by writing an expired blob and clearing - /// the cache entry. The file is left on disk (rewriteable) so concurrent - /// readers don't race with unlink. + /// Ends the session: its file is removed and its cache entry dropped. A + /// request reading it at the same moment finds no session, as it should. Future deleteSession(String sessionId) async { final hashedId = _makeHash(sessionId).toString(); _cache.remove(hashedId); - final file = File('$sessionPath/$hashedId'); - if (await file.exists()) { - final raf = await file.open(mode: FileMode.write); - try { - final expiration = DateTime.now().toUtc().millisecondsSinceEpoch - 1; - final content = await VaniaEncryption.encryptString( - json.encode({'data': {}, 'expiration': expiration}), - _secretKey, - ); - await raf.writeFrom(utf8.encode(content)); - } finally { - try { - await raf.unlock(); - } catch (_) {} - await raf.close(); - } + try { + await File('$sessionPath/$hashedId').delete(); + } catch (_) { + // Already gone. } } diff --git a/packages/core/lib/src/http/session/session_manager.dart b/packages/core/lib/src/http/session/session_manager.dart index 403e342..71009c5 100644 --- a/packages/core/lib/src/http/session/session_manager.dart +++ b/packages/core/lib/src/http/session/session_manager.dart @@ -4,6 +4,7 @@ import 'package:crypto/crypto.dart'; import 'package:vania/src/http/request/request_scope.dart'; import 'package:vania/src/utils/functions.dart'; import 'package:vania/env.dart' show env; +import 'flash_messages.dart'; import 'session_file_store.dart'; /// Session lifecycle owner. @@ -22,9 +23,6 @@ class SessionManager { Map get allSessions => currentRequestScope?.sessionData ?? _fallbackSessions; - final Duration _sessionLifeTime = Duration( - seconds: env('SESSION_LIFETIME', 9000), - ); bool secureSession = env('SECURE_SESSION', true); String _generateSessionId() { @@ -40,15 +38,18 @@ class SessionManager { (cookie) => cookie.name == 'XSRF-TOKEN', orElse: () => Cookie('XSRF-TOKEN', ''), ); - if (requestCookie.value.isEmpty) { + final storedToken = allSessions['x_csrf_token']?.toString() ?? ''; + final storedIv = allSessions['x_csrf_iv']?.toString() ?? ''; + if (requestCookie.value.isEmpty || + storedToken.isEmpty || + storedIv.isEmpty) { await _generateNewCsrfToken(response); } else { - final storedToken = allSessions['x_csrf_token']; - if (storedToken == null || storedToken.toString().isEmpty) { - await _generateNewCsrfToken(response); - } else { - _setCsrfToken(storedToken.toString()); - } + _setCsrfToken(storedToken); + // Sent again with every page, as the session cookie is, so the two + // expire together: a form left open as long as the session lasts + // can still be sent. + response.cookies.add(_xsrfCookie(storedToken, storedIv)); } } @@ -67,16 +68,19 @@ class SessionManager { await setSession('x_csrf_token', token); await setSession('x_csrf_iv', iv); _setCsrfToken(token); - final cookieValue = _computeCsrfCookieValue(token, iv); - final cookie = Cookie('XSRF-TOKEN', cookieValue) - ..expires = DateTime.now().add(const Duration(seconds: 9000)) - ..sameSite = SameSite.lax - ..secure = secureSession - ..path = '/' - ..httpOnly = true; - response.cookies.add(cookie); + response.cookies.add(_xsrfCookie(token, iv)); } + /// The cookie that pairs the browser with [token]. It lasts as long as + /// the session the token is kept in. + Cookie _xsrfCookie(String token, String iv) => + Cookie('XSRF-TOKEN', _computeCsrfCookieValue(token, iv)) + ..expires = DateTime.now().add(sessionLifetime) + ..sameSite = SameSite.lax + ..secure = secureSession + ..path = '/' + ..httpOnly = true; + String _computeCsrfCookieValue(String token, String iv) { final hmac = Hmac(sha512, utf8.encode(iv)); final Digest digest = hmac.convert(utf8.encode(token)); @@ -88,34 +92,82 @@ class SessionManager { /// Starts a session for [request], writing the session cookie on /// [response] and hydrating [RequestScope.sessionData] from disk. Future sessionStart(HttpRequest request, HttpResponse response) async { - final cookie = request.cookies.firstWhere( - (c) => c.name == sessionKey, - orElse: () => Cookie(sessionKey, _generateSessionId()), - ); - final sessionId = cookie.value; - - response.cookies.add( - Cookie(sessionKey, sessionId) - ..httpOnly = true - ..secure = secureSession - ..path = '/' - ..sameSite = SameSite.lax - ..expires = DateTime.now().add(_sessionLifeTime), - ); + final presented = request.cookies + .where((c) => c.name == sessionKey && c.value.isNotEmpty) + .firstOrNull + ?.value; + final stored = presented == null + ? null + : await SessionFileStore().retrieveSessionEntry(presented); + // An id with no session behind it — one the server never issued, or + // one whose session has ended — is not taken up. Otherwise whoever put + // it in the browser would know the id of the session its user is about + // to sign in to. + final sessionId = stored != null ? presented! : _generateSessionId(); + _setSessionCookie(response, sessionId); final scope = currentRequestScope; if (scope != null) { scope.sessionId = sessionId; - scope.sessionData = - await SessionFileStore().retrieveSession(sessionId) ?? {}; + scope.sessionData = stored?.data ?? {}; + scope.sessionLoaded = true; + // A session in use is kept alive. Rewriting it on every request + // would cost a write each time, so it is renewed once past half its + // lifetime: at most two writes per lifetime, and an active user is + // never signed out. + if (stored != null && + stored.expiresAt.difference(DateTime.now()) < sessionLifetime ~/ 2) { + (await _changesFor(scope))?.renew = true; + } + await FlashMessages.restore(scope, this); } else { - _fallbackSessions = - await SessionFileStore().retrieveSession(sessionId) ?? {}; + _fallbackSessions = stored?.data ?? {}; } await createXsrfToken(request, response); } + void _setSessionCookie(HttpResponse response, String sessionId) { + response.cookies + ..removeWhere((c) => c.name == sessionKey) + ..add( + Cookie(sessionKey, sessionId) + ..httpOnly = true + ..secure = secureSession + ..path = '/' + ..sameSite = SameSite.lax + ..expires = DateTime.now().add(sessionLifetime), + ); + } + + /// Moves the session to a new id, keeping its data, and ends the old one. + /// + /// Call it when a user signs in, and whenever their privileges change. + /// An id someone else may have learned before — planted in the browser, + /// or read off an unencrypted connection — then leads nowhere, instead + /// of into the signed-in session. + Future regenerateSession() async { + final scope = currentRequestScope; + final oldId = getSessionId(); + if (scope == null || oldId == null) return; + await _load(scope, oldId); + + // Whatever this request changed so far is in sessionData already; it + // all goes to the new id instead. + final pending = scope.attributes[_SessionChanges]; + if (pending is _SessionChanges) pending.discarded = true; + + final newId = _generateSessionId(); + scope.sessionId = newId; + scope.attributes.remove(_Unchecked); + _setSessionCookie(scope.request.response, newId); + final changes = _SessionChanges(scope, newId, replaces: oldId) + ..cleared = true + ..values.addAll(scope.sessionData); + scope.attributes[_SessionChanges] = changes; + scope.beforeResponse(changes.save); + } + /// Resolves the session id for the current request from the /// zone-scoped scope, or from the request cookies as a fallback. String? getSessionId() { @@ -127,29 +179,20 @@ class SessionManager { orElse: () => Cookie(sessionKey, ''), ); scope.sessionId = cookie.value.isEmpty ? null : cookie.value; + // Taken from the cookie as it is: whether the server issued it is + // found out when the session is read. + if (scope.sessionId != null) scope.attributes[_Unchecked] = true; return scope.sessionId; } return null; } Future getSession(String key) async { - final bag = allSessions; - if (bag.isEmpty) { - final sessionId = getSessionId(); - if (sessionId != null) { - final hydrated = - await SessionFileStore().retrieveSession(sessionId) ?? {}; - final scope = currentRequestScope; - if (scope != null) { - scope.sessionData = hydrated; - } else { - _fallbackSessions = hydrated; - } - } - } + final scope = currentRequestScope; + final sessionId = getSessionId(); + if (scope != null && sessionId != null) await _load(scope, sessionId); - final current = allSessions; - final raw = current[key]; + final raw = allSessions[key]; if (raw == null) return null as T; final typeName = T.toString(); @@ -159,51 +202,123 @@ class SessionManager { return raw; } + /// Sets [key] for the rest of this request, and in the store before the + /// response is written. Future setSession(String key, dynamic value) async { - final sessionId = getSessionId(); - if (sessionId == null) return; - - Map? session = await SessionFileStore().retrieveSession( - sessionId, - ); - if (session != null) { - session[key] = value; - } else { - session = {key: value}; - } - final scope = currentRequestScope; - if (scope != null) { - scope.sessionData = session; - } else { - _fallbackSessions = session; - } - await SessionFileStore().storeSession(sessionId, session); + final changes = await _changesFor(currentRequestScope); + if (changes == null) return; + changes.scope.sessionData[key] = value; + changes.values[key] = value; } Future deleteSession(String key) async { - final sessionId = getSessionId(); - if (sessionId == null) return; - final session = await SessionFileStore().retrieveSession(sessionId); - if (session == null) return; - session.remove(key); - final scope = currentRequestScope; - if (scope != null) { - scope.sessionData = session; - } else { - _fallbackSessions = session; - } - await SessionFileStore().storeSession(sessionId, session); + final changes = await _changesFor(currentRequestScope); + if (changes == null) return; + changes.scope.sessionData.remove(key); + changes.values[key] = _removed; } Future destroyAllSessions() async { - final sessionId = getSessionId(); - if (sessionId == null) return; - final scope = currentRequestScope; - if (scope != null) { - scope.sessionData = {}; - } else { - _fallbackSessions = {}; + final changes = await _changesFor(currentRequestScope); + if (changes == null) return; + changes.scope.sessionData = {}; + changes.values.clear(); + changes.cleared = true; + } + + Future _load(RequestScope scope, String sessionId) async { + if (scope.sessionLoaded) return; + final stored = await SessionFileStore().retrieveSession(sessionId); + scope.sessionData = stored ?? {}; + scope.sessionLoaded = true; + if (stored != null) scope.attributes.remove(_Unchecked); + } + + /// The pending changes for this request's session, registering their + /// save on first use. Null outside a request or without a session. + Future<_SessionChanges?> _changesFor(RequestScope? scope) async { + var sessionId = getSessionId(); + if (scope == null || sessionId == null) return null; + final existing = scope.attributes[_SessionChanges]; + if (existing is _SessionChanges) return existing; + + await _load(scope, sessionId); + if (scope.attributes.remove(_Unchecked) != null) { + // The cookie names no session. Writing one under that id would let + // whoever put it in the browser into the session — a request that + // never went through [sessionStart], such as a `fetch` to an API + // route, takes the cookie as it is. A new id is issued instead. + sessionId = _generateSessionId(); + try { + _setSessionCookie(scope.request.response, sessionId); + } on StateError { + // The response has gone out, so the client could not be told the + // new id: there is nothing to save the changes under. + return null; + } + scope.sessionId = sessionId; + } + final changes = _SessionChanges(scope, sessionId); + scope.attributes[_SessionChanges] = changes; + scope.beforeResponse(changes.save); + return changes; + } +} + +/// Marks a key deleted in [_SessionChanges.values]. +const Object _removed = Object(); + +/// Key of the [RequestScope.attributes] flag set while the request's +/// session id is one read from its cookie and not yet found in the store. +abstract final class _Unchecked {} + +/// The session writes one request makes, saved together once. +/// +/// Only the keys this request touched are written, onto the stored +/// session as it stands at save time. Saving the request's whole copy +/// instead would undo whatever a concurrent request from the same client +/// wrote in the meantime — a login, say, made in another tab. +class _SessionChanges { + final RequestScope scope; + final String sessionId; + + /// Changed keys in the order they were changed; [_removed] for deletes. + final Map values = {}; + + /// Whether the session was emptied before [values] were made. + bool cleared = false; + + /// Whether to write the session even with nothing changed, to give it a + /// fresh expiry. + bool renew = false; + + /// Set when the session moved to a new id before this could be saved. + bool discarded = false; + + /// The id this session moves from; that session ends once this is saved. + final String? replaces; + + _SessionChanges(this.scope, this.sessionId, {this.replaces}); + + Future save() async { + // Changes made from here on start a fresh set with a save of its own. + if (identical(scope.attributes[_SessionChanges], this)) { + scope.attributes.remove(_SessionChanges); } - await SessionFileStore().storeSession(sessionId, {}); + if (discarded || (!cleared && !renew && values.isEmpty)) return; + final store = SessionFileStore(); + final stored = cleared ? null : await store.retrieveSession(sessionId); + final session = stored ?? {}; + values.forEach((key, value) { + if (identical(value, _removed)) { + session.remove(key); + } else { + session[key] = value; + } + }); + if (stored == null && !cleared && session.isEmpty) return; + await store.storeSession(sessionId, session); + final old = replaces; + if (old != null) await store.deleteSession(old); } } diff --git a/packages/core/lib/src/logger/logger.dart b/packages/core/lib/src/logger/logger.dart index 70e2d37..03948a7 100644 --- a/packages/core/lib/src/logger/logger.dart +++ b/packages/core/lib/src/logger/logger.dart @@ -13,6 +13,18 @@ class Logger { static const INFO = 'INFO'; static const DEBUG = 'DEBUG'; + /// One open sink per log file, kept for the life of the process. + /// + /// Opening a file per entry costs a directory check and a file + /// descriptor each time, and under a burst of errors the descriptors + /// pile up faster than they close. + static final Map _sinks = {}; + + /// Appends [content] to `storage/logs/.log`. + /// + /// Never throws: logging is called from error handlers, and a failure + /// here must not turn one error into another. An entry that cannot be + /// written goes to stderr instead. static void log( String content, { String type = INFO, @@ -20,21 +32,53 @@ class Logger { }) { final now = DateTime.now(); - final directory = Directory(storagePath('logs')); - if (!directory.existsSync()) { - directory.createSync(recursive: true); - } - - final logFile = File('${directory.path}/$fileName.log'); - final fsSink = logFile.openWrite(mode: FileMode.append); - var text = "[${now.year}-${now.month.toString().padLeft(2, '0')}-${now.day.toString().padLeft(2, '0')} ${now.hour.toString().padLeft(2, '0')}:${now.minute.toString().padLeft(2, '0')}:${now.second.toString().padLeft(2, '0')}]"; text += ' $type: '; text += content; text += '\n'; - fsSink.write(text); - fsSink.close(); + try { + _sinkFor(fileName).write(text); + } catch (_) { + stderr.write(text); + } + } + + static IOSink _sinkFor(String fileName) { + final existing = _sinks[fileName]; + if (existing != null) return existing; + + final directory = Directory(storagePath('logs')); + if (!directory.existsSync()) { + directory.createSync(recursive: true); + } + final sink = File( + '${directory.path}/$fileName.log', + ).openWrite(mode: FileMode.append); + _sinks[fileName] = sink; + + // A failed write surfaces here, asynchronously. Left unhandled it + // would stop the isolate; instead the sink is dropped so the next + // entry opens the file afresh. + sink.done.catchError((Object e) { + if (identical(_sinks[fileName], sink)) _sinks.remove(fileName); + stderr.writeln('Logger: cannot write $fileName.log: $e'); + }); + return sink; + } + + /// Writes out every buffered entry and closes the open log files. + /// + /// Call before the process exits so buffered entries are not lost. + /// Logging afterwards reopens the files. + static Future flush() async { + final sinks = _sinks.values.toList(); + _sinks.clear(); + for (final sink in sinks) { + try { + await sink.close(); + } catch (_) {} + } } } diff --git a/packages/core/lib/src/route/middleware/csrf_middleware.dart b/packages/core/lib/src/route/middleware/csrf_middleware.dart index b7090f7..6c46df5 100644 --- a/packages/core/lib/src/route/middleware/csrf_middleware.dart +++ b/packages/core/lib/src/route/middleware/csrf_middleware.dart @@ -23,10 +23,19 @@ class CsrfMiddleware extends Middleware { final SessionManager _sessionManager = IoCContainer() .resolve(); + /// The methods that change state, and so need a token. The request + /// handler attaches this middleware to routes by the same set, so the + /// two cannot disagree about which requests are checked. + static const Set protectedMethods = { + 'post', + 'put', + 'patch', + 'delete', + }; + @override Future handle(Request req) async { - final method = req.method?.toLowerCase(); - if (method != 'post' && method != 'put' && method != 'patch') return; + if (!protectedMethods.contains(req.method?.toLowerCase())) return; // `api/*` is excluded by default: API routes are normally // token-authenticated, where CSRF does not apply. Apps that @@ -105,21 +114,25 @@ class CsrfMiddleware extends Middleware { return value; } + /// Whether [path] is one `csrf_except` leaves out. + /// + /// A pattern names a path and everything under it — `webhooks` covers + /// `webhooks/stripe`, not `webhooks-admin`. In a pattern with `*`, the + /// `*` matches anything and every other character only itself. bool _isUrlExcluded(String path, List csrfExcept) { - final cleanPath = path.startsWith('/') ? path.substring(1) : path; + final cleanPath = (path.startsWith('/') ? path.substring(1) : path) + .toLowerCase(); for (final pattern in csrfExcept) { - final cleanPattern = pattern.startsWith('/') - ? pattern.substring(1) - : pattern; + final cleanPattern = + (pattern.startsWith('/') ? pattern.substring(1) : pattern) + .toLowerCase(); if (cleanPattern.contains('*')) { - final regexStr = cleanPattern - .replaceAll('*', '.*') - .replaceAll('/', '\\/'); - final regex = RegExp('^$regexStr\$', caseSensitive: false); + final regex = RegExp( + '^${RegExp.escape(cleanPattern).replaceAll(r'\*', '.*')}\$', + ); if (regex.hasMatch(cleanPath)) return true; - } else if (cleanPath.toLowerCase().startsWith( - cleanPattern.toLowerCase(), - )) { + } else if (cleanPath == cleanPattern || + cleanPath.startsWith('$cleanPattern/')) { return true; } } diff --git a/packages/core/lib/src/route/middleware/throttle.dart b/packages/core/lib/src/route/middleware/throttle.dart index 3cabfb2..a4fa446 100644 --- a/packages/core/lib/src/route/middleware/throttle.dart +++ b/packages/core/lib/src/route/middleware/throttle.dart @@ -7,12 +7,23 @@ import 'package:vania/env.dart' show env; import '../../exception/throttle_exception.dart'; import '../throttle_requests.dart'; +/// `APP_ENV` values that count as development for +/// [Throttle.bypassInDevelopment]: the documented `local`, and +/// `development`. +const Set _developmentEnvironments = {'local', 'development'}; + class Throttle extends Middleware { final int maxAttempts; final Duration duration; final bool includeUserIdentifier; final String? customMessage; final Map? headers; + + /// Skips the limit while `APP_ENV` is `local` or `development`. + /// + /// Off unless asked for: `local` is also what a `.env` copied from a + /// development machine says, and a production app that kept it would + /// lose its rate limits without a word. final bool bypassInDevelopment; late final ThrottleRequests _throttle; @@ -23,7 +34,7 @@ class Throttle extends Middleware { this.includeUserIdentifier = false, this.customMessage, this.headers, - this.bypassInDevelopment = true, + this.bypassInDevelopment = false, }) { _throttle = ThrottleRequests(maxAttempts: maxAttempts, duration: duration); } @@ -33,16 +44,23 @@ class Throttle extends Middleware { // Fails closed: an unset APP_ENV is treated as production, so an app // that never sets it still gets rate limiting. if (bypassInDevelopment && - env('APP_ENV', 'production') == 'development') { + _developmentEnvironments.contains( + env('APP_ENV', 'production'), + )) { return; } final String identifier = await _getRequestIdentifier(req); - final remaining = _throttle.remainingAttempts(identifier); + final allowed = _throttle.request(identifier); - _addRateLimitHeaders(req.response, remaining); + // Counted after this request, so the last one allowed says 0 left. + _addRateLimitHeaders( + req.response, + _throttle.remainingAttempts(identifier), + _throttle.resetAt(identifier), + ); - if (!_throttle.request(identifier)) { + if (!allowed) { final retryAfter = _throttle.retryAfter(identifier); throw ThrottleException( message: customMessage ?? 'Too Many Requests. Please try again later.', @@ -65,12 +83,13 @@ class Throttle extends Middleware { return parts.join(':'); } - void _addRateLimitHeaders(HttpResponse response, int remaining) { + void _addRateLimitHeaders( + HttpResponse response, + int remaining, + DateTime resetAt, + ) { response.headers.add('X-RateLimit-Limit', maxAttempts.toString()); response.headers.add('X-RateLimit-Remaining', remaining.toString()); - response.headers.add( - 'X-RateLimit-Reset', - _throttle.resetTime().toIso8601String(), - ); + response.headers.add('X-RateLimit-Reset', resetAt.toIso8601String()); } } diff --git a/packages/core/lib/src/route/route_data.dart b/packages/core/lib/src/route/route_data.dart index 0cc211f..832ae41 100644 --- a/packages/core/lib/src/route/route_data.dart +++ b/packages/core/lib/src/route/route_data.dart @@ -14,6 +14,10 @@ class RouteData { Map? paramTypes; Map? regex; + /// The media types a `QUERY` route takes, from `acceptQuery`; null to + /// take any. + List? acceptQuery; + RouteData({ required this.method, required this.path, @@ -27,6 +31,7 @@ class RouteData { this.paramTypes, this.name, this.regex, + this.acceptQuery, }); @override diff --git a/packages/core/lib/src/route/route_handler.dart b/packages/core/lib/src/route/route_handler.dart index 5e63e83..864b157 100644 Binary files a/packages/core/lib/src/route/route_handler.dart and b/packages/core/lib/src/route/route_handler.dart differ diff --git a/packages/core/lib/src/route/router.dart b/packages/core/lib/src/route/router.dart index 7c07cdf..57f8f26 100644 --- a/packages/core/lib/src/route/router.dart +++ b/packages/core/lib/src/route/router.dart @@ -16,6 +16,12 @@ class Router { final List _groupMiddleware = []; final List _routes = []; + /// What the last registration added: one route, or one per method for + /// [any]. Everything chained after it — middleware, constraints, a name — + /// applies to all of these, so no method of an `any` route is left + /// without the middleware meant to guard it. + List _lastAdded = const []; + static final RegExp _requestVarRegex = RegExp(r'Closure: \(([^)]*)\) =>'); static final RegExp _closureStartRegex = RegExp(r'Closure: \('); @@ -75,17 +81,17 @@ class Router { } final normalizedPath = _normalizePath(path); - _routes.add( - RouteData( - method: method.name, - path: normalizedPath, - action: action, - prefix: _prefix, - paramTypes: paramTypes, - regex: regex, - hasRequest: hasRequest, - ), + final route = RouteData( + method: method.name, + path: normalizedPath, + action: action, + prefix: _prefix, + paramTypes: paramTypes, + regex: regex, + hasRequest: hasRequest, ); + _routes.add(route); + _lastAdded = [route]; return this; } @@ -106,79 +112,90 @@ class Router { } Router middleware([List? middleware]) { - if (middleware != null && _routes.isNotEmpty) { - _routes.last.preMiddleware = [ - ..._routes.last.preMiddleware, - ...middleware, - ]; + if (middleware != null) { + for (final route in _lastAdded) { + route.preMiddleware = [...route.preMiddleware, ...middleware]; + } } return this; } Router prefix([String? prefix]) { - if (prefix != null && _routes.isNotEmpty) { - final route = _routes.last; - final basePath = route.path.startsWith('/') - ? route.path.substring(1) - : route.path; - - route.path = prefix.endsWith("/") - ? "$prefix$basePath" - : "$prefix/$basePath"; + if (prefix != null) { + for (final route in _lastAdded) { + final basePath = route.path.startsWith('/') + ? route.path.substring(1) + : route.path; + + route.path = prefix.endsWith("/") + ? "$prefix$basePath" + : "$prefix/$basePath"; + } } return this; } Router name([String? name]) { - if (name != null && _routes.isNotEmpty) { - _routes.last.name = name; + if (name != null) { + for (final route in _lastAdded) { + route.name = name; + } } return this; } Router domain([String? domain]) { - if (domain != null && _routes.isNotEmpty) { - _routes.last.domain = domain; + if (domain != null) { + for (final route in _lastAdded) { + route.domain = domain; + } + } + return this; + } + + /// The media types this `QUERY` route takes, such as + /// `application/json`; `type/*` and `*/*` match a family or anything. A + /// request with another type is answered with a 415, and the list is + /// advertised in the `Accept-Query` header. + Router acceptQuery(List mediaTypes) { + final accepted = [for (final type in mediaTypes) type.trim().toLowerCase()]; + for (final route in _lastAdded) { + route.acceptQuery = accepted; } return this; } Router whereInt(String paramName) { - if (_routes.isNotEmpty) { - _routes.last.paramTypes ??= {}; - _routes.last.paramTypes![paramName] = int; + for (final route in _lastAdded) { + (route.paramTypes ??= {})[paramName] = int; } return this; } Router whereString(String paramName) { - if (_routes.isNotEmpty) { - _routes.last.paramTypes ??= {}; - _routes.last.paramTypes![paramName] = String; + for (final route in _lastAdded) { + (route.paramTypes ??= {})[paramName] = String; } return this; } Router whereDouble(String paramName) { - if (_routes.isNotEmpty) { - _routes.last.paramTypes ??= {}; - _routes.last.paramTypes![paramName] = double; + for (final route in _lastAdded) { + (route.paramTypes ??= {})[paramName] = double; } return this; } Router whereBool(String paramName) { - if (_routes.isNotEmpty) { - _routes.last.paramTypes ??= {}; - _routes.last.paramTypes![paramName] = bool; + for (final route in _lastAdded) { + (route.paramTypes ??= {})[paramName] = bool; } return this; } Router where(String paramName, String regex) { - if (_routes.isNotEmpty) { - _routes.last.regex ??= {}; - _routes.last.regex![paramName] = regex; + for (final route in _lastAdded) { + (route.regex ??= {})[paramName] = regex; } return this; } @@ -222,25 +239,27 @@ class Router { static Router propfind(String path, Function action) => _addRoute(HttpRequestMethod.propfind, path, action); + /// A `QUERY` route (RFC 10008): a safe, idempotent read like `GET`, with + /// the query in the request content instead of the URL — so it is not + /// limited in length, and is less often written to logs on the way. + /// + /// The request must say what its content is with `Content-Type`; one + /// that does not, or whose content does not parse as that type, is + /// answered with a 400. Content the framework does not parse into + /// fields, such as `application/sql`, is in `Request.rawBody`. + static Router query(String path, Function action) => + _addRoute(HttpRequestMethod.query, path, action); + + /// A route for every HTTP method. Anything chained after it applies to + /// all of them. static Router any(String path, Function action) { final router = Router(); - - final currentPrefix = router._prefix; - - for (HttpRequestMethod method in HttpRequestMethod.values) { - final routeData = RouteData( - method: method.name, - path: router._normalizePath(path), - action: action, - prefix: currentPrefix, - hasRequest: router._getRequestVar(action.toString()), - preMiddleware: List.from(router._groupMiddleware), - domain: router._groupDomain, - ); - - router._routes.add(routeData); + final added = []; + for (final method in HttpRequestMethod.values) { + _addRoute(method, path, action); + added.addAll(router._lastAdded); } - + router._lastAdded = added; return router; } @@ -250,7 +269,7 @@ class Router { String? prefix, List? middleware, String? domain, - String regex = r'\d+(.\d+)?', + String regex = r'\d+(\.\d+)?', }) { Router.get( path, @@ -288,6 +307,50 @@ class Router { ).middleware(middleware).domain(domain).prefix(prefix).where('id', regex); } + /// The routes of a JSON API resource: `index`, `store`, `show`, + /// `update` (under both PUT and PATCH) and `destroy`. + /// + /// [resource] without `create` and `edit`, which only serve HTML forms, + /// so the controller does not need them. + static void apiResource( + String path, + dynamic controller, { + String? prefix, + List? middleware, + String? domain, + String regex = r'\d+(\.\d+)?', + }) { + Router.get( + path, + controller.index, + ).middleware(middleware).domain(domain).prefix(prefix); + + Router.post( + path, + controller.store, + ).middleware(middleware).domain(domain).prefix(prefix); + + Router.get( + "$path/{id}", + controller.show, + ).middleware(middleware).domain(domain).prefix(prefix).where('id', regex); + + Router.put( + "$path/{id}", + controller.update, + ).middleware(middleware).domain(domain).prefix(prefix).where('id', regex); + + Router.patch( + "$path/{id}", + controller.update, + ).middleware(middleware).domain(domain).prefix(prefix).where('id', regex); + + Router.delete( + "$path/{id}", + controller.destroy, + ).middleware(middleware).domain(domain).prefix(prefix).where('id', regex); + } + static void group( Function callback, { String? prefix = '', @@ -321,14 +384,18 @@ class Router { if (middleware.isNotEmpty) { router._groupMiddleware.addAll(middleware); } - callback(); - - router._groupDomain = previousDomain; - router._groupPrefix = previousPrefix; - - router._groupMiddleware - ..clear() - ..addAll(previousMiddleware); + try { + callback(); + } finally { + // Restored even when the callback fails, so the group's prefix and + // middleware never reach routes registered after it. + router._groupDomain = previousDomain; + router._groupPrefix = previousPrefix; + + router._groupMiddleware + ..clear() + ..addAll(previousMiddleware); + } } static String _joinPrefixes(String basePrefix, String newPrefix) { diff --git a/packages/core/lib/src/route/set_static_path.dart b/packages/core/lib/src/route/set_static_path.dart index e73aa8c..bff23d7 100644 --- a/packages/core/lib/src/route/set_static_path.dart +++ b/packages/core/lib/src/route/set_static_path.dart @@ -37,11 +37,13 @@ set staticPublicRoot(String root) { /// at most once per [_publicDirRecheck] window and every GET short-circuits /// on a cached bool until then. Future setStaticPathAsync(HttpRequest req) async { - // Only GET serves static content. Compared against the raw method, - // which may arrive in either case, without allocating a copy. + // Only GET and HEAD serve static content. Compared against the raw + // method, which may arrive in either case, without allocating a copy. final m = req.method; - if (m.length != 3 || - !(m == 'GET' || m == 'get' || m.toLowerCase() == 'get')) { + final isHead = m == 'HEAD' || (m.length == 4 && m.toLowerCase() == 'head'); + if (!isHead && + (m.length != 3 || + !(m == 'GET' || m == 'get' || m.toLowerCase() == 'get'))) { return false; } @@ -52,11 +54,11 @@ Future setStaticPathAsync(HttpRequest req) async { if (!_publicDirAvailable()) return false; - final String routePath = rawPath == '/' || rawPath.isEmpty + final String? routePath = rawPath == '/' || rawPath.isEmpty ? 'index.html' - : _stripLeadingSlash(rawPath); + : _decodeFilePath(_stripLeadingSlash(rawPath)); - if (_hasHiddenSegment(routePath)) return false; + if (routePath == null || _hasHiddenSegment(routePath)) return false; final file = File('$_publicRoot/$routePath'); // Most requests to an app that also serves an API are routed, not static, @@ -105,7 +107,8 @@ Future setStaticPathAsync(HttpRequest req) async { : ContentType('application', 'octet-stream'); res.headers.contentLength = length; - await res.addStream(file.openRead()); + // A HEAD gets the headers alone, so there is no reason to read the file. + if (!isHead) await res.addStream(file.openRead()); await res.close(); return true; } @@ -117,10 +120,7 @@ Future setStaticPathAsync(HttpRequest req) async { /// repeat visit into a 304 with no body, while never serving a stale asset. /// Apps that fingerprint their asset names can trade that for an immutable /// `max-age` through `STATIC_CACHE_CONTROL`. -final String _cacheControl = env( - 'STATIC_CACHE_CONTROL', - 'no-cache', -); +final String _cacheControl = env('STATIC_CACHE_CONTROL', 'no-cache'); DateTime _truncateToSeconds(DateTime t) => DateTime.utc(t.year, t.month, t.day, t.hour, t.minute, t.second); @@ -203,6 +203,34 @@ void resetStaticPathCache() { _publicDirClock = null; } +/// [p] as the file path it names: a URL arrives percent-encoded, while a +/// file is named in text — `my%20file.css` is `my file.css`. +/// +/// Each segment is decoded on its own, and a segment that decodes to +/// anything able to change the path — a `/`, a `\`, a NUL — names no +/// file: `%2F` and `%5C` are separators in disguise. A decoded `..` or +/// `.env` begins with a dot, which the caller then turns away like any +/// hidden segment. A raw `\` is refused outright, being a separator on +/// Windows. Null when the path names no file. +String? _decodeFilePath(String p) { + if (p.contains('\\')) return null; + if (!p.contains('%')) return p; + final decoded = []; + for (final segment in p.split('/')) { + final String text; + try { + text = Uri.decodeComponent(segment); + } catch (_) { + return null; + } + if (text.contains('/') || text.contains('\\') || text.contains('\x00')) { + return null; + } + decoded.add(text); + } + return decoded.join('/'); +} + /// True when any segment of [p] starts with a dot. bool _hasHiddenSegment(String p) { var start = 0; diff --git a/packages/core/lib/src/route/throttle_requests.dart b/packages/core/lib/src/route/throttle_requests.dart index c1bc00f..c20068e 100644 --- a/packages/core/lib/src/route/throttle_requests.dart +++ b/packages/core/lib/src/route/throttle_requests.dart @@ -63,6 +63,16 @@ class ThrottleRequests { return DateTime.now().add(duration); } + /// When [identifier]'s current window ends and its count starts over. + DateTime resetAt(String identifier) { + final data = _requests[identifier]; + final now = DateTime.now(); + if (data == null || now.difference(data.firstAttempt) >= duration) { + return now.add(duration); + } + return data.firstAttempt.add(duration); + } + void _maybeCleanup() { _sinceLastCleanup++; if (_sinceLastCleanup < _cleanupEveryNRequests && diff --git a/packages/core/lib/src/server/base_http_server.dart b/packages/core/lib/src/server/base_http_server.dart index 3ca80d0..ea25913 100644 --- a/packages/core/lib/src/server/base_http_server.dart +++ b/packages/core/lib/src/server/base_http_server.dart @@ -1,3 +1,4 @@ +import 'dart:async'; import 'dart:io'; import '../route/route_inspector.dart'; import 'package:args/args.dart'; @@ -5,13 +6,23 @@ import 'package:vania/src/http/request/request_handler.dart'; import 'package:vania/env.dart' show env; import '../ioc_container.dart'; +import '../logger/logger.dart'; import 'initialize_config.dart'; +import 'server_isolates.dart'; class BaseHttpServer { final Map config; final List args; - BaseHttpServer({required this.config, this.args = const []}); + /// True in the extra isolates started for `APP_ISOLATES`, which serve + /// alongside the main one and so do not announce themselves. + final bool isWorker; + + BaseHttpServer({ + required this.config, + this.args = const [], + this.isWorker = false, + }); HttpServer? httpServer; @@ -65,6 +76,15 @@ class BaseHttpServer { // table, so what it prints is what the app would actually serve. dumpRoutesIfRequested(); + // Isolates serving the same port must all bind it shared. + final shared = env('APP_SHARED', false) || serverIsolateCount > 1; + + // Connections the OS queues while the server is busy accepting. A + // burst beyond it is refused, so the OS's own default — often 128 — + // is too small for real traffic. The OS caps it in turn + // (`net.core.somaxconn` on Linux, `kern.ipc.somaxconn` on macOS). + final backlog = env('HTTP_BACKLOG', 1024); + if (env('APP_SECURE', false)) { String certificateChain = env('APP_CERTIFICATE'); String serverKey = env('APP_PRIVATE_KEY'); @@ -78,20 +98,38 @@ class BaseHttpServer { host, port, context, - shared: env('APP_SHARED', false), + backlog: backlog, + shared: shared, ); } else { httpServer = await HttpServer.bind( host, port, - shared: env('APP_SHARED', false), + backlog: backlog, + shared: shared, ); } - httpServer?.listen(IoCContainer().resolve().handle); - - if (env('APP_DEBUG')) { - stderr.writeln('Server started on http://127.0.0.1:$port'); + // Seconds an idle keep-alive connection is held open; 0 never + // closes it. 120 matches dart:io's own default. + final idleSeconds = env('HTTP_IDLE_TIMEOUT', 120); + httpServer!.idleTimeout = idleSeconds > 0 + ? Duration(seconds: idleSeconds) + : null; + + httpServer?.listen( + IoCContainer().resolve().handle, + // An unhandled error on the server stream would stop the isolate; + // a single bad connection is not worth that. + onError: (Object e, StackTrace stack) => + Logger.log('HTTP server error: $e\n$stack', type: Logger.ERROR), + ); + + if (!isWorker && env('APP_DEBUG')) { + final isolates = serverIsolateCount > 1 + ? ' ($serverIsolateCount isolates)' + : ''; + stderr.writeln('Server started on http://127.0.0.1:$port$isolates'); } return httpServer!; } catch (e) { @@ -99,4 +137,21 @@ class BaseHttpServer { rethrow; } } + + /// Stops accepting connections and lets in-flight requests finish. + /// + /// Idle keep-alive connections close at once and busy ones as soon as + /// their response completes. Whatever is still open after [timeout] — + /// a stalled client, a long-lived stream — is closed regardless. + Future shutdown({required Duration timeout}) async { + final server = httpServer; + if (server == null) return; + + unawaited(server.close()); + final waited = Stopwatch()..start(); + while (server.connectionsInfo().total > 0 && waited.elapsed < timeout) { + await Future.delayed(const Duration(milliseconds: 50)); + } + await server.close(force: true); + } } diff --git a/packages/core/lib/src/server/server_isolates.dart b/packages/core/lib/src/server/server_isolates.dart new file mode 100644 index 0000000..dacddbf --- /dev/null +++ b/packages/core/lib/src/server/server_isolates.dart @@ -0,0 +1,50 @@ +import 'dart:io'; + +import 'package:vania/env.dart' show env; + +/// How many isolates serve HTTP, from `APP_ISOLATES`: a number, or `auto` +/// for one per CPU core. Defaults to 1. +/// +/// Each isolate has its own memory, so anything kept in memory is per +/// isolate: rate-limit counters, WebSocket rooms, and whatever an app +/// holds in a static field. The session cache is switched off when this +/// is above 1, since one isolate would otherwise serve a session that +/// another has since changed. +final int serverIsolateCount = _resolveIsolateCount(); + +int _resolveIsolateCount() { + final raw = env('APP_ISOLATES', null)?.trim().toLowerCase(); + if (raw == null || raw.isEmpty) return 1; + if (raw == 'auto') return Platform.numberOfProcessors; + final count = int.tryParse(raw); + return (count == null || count < 1) ? 1 : count; +} + +/// The longest a crashed server isolate waits to be replaced. +const Duration maxIsolateRestartDelay = Duration(minutes: 1); + +/// When to replace a server isolate that exited, given [retryAfter], the +/// wait set aside for it: the [wait] before starting its replacement, and +/// the [next] wait, should that one exit too. +/// +/// One that dies soon after starting is likely to die again — a failing +/// boot, say — so each such death doubles the wait, up to +/// [maxIsolateRestartDelay]. One that ran a while is replaced at once. +({Duration wait, Duration next}) isolateRestartDelay( + Duration retryAfter, { + required bool crashedEarly, +}) { + if (!crashedEarly) { + return (wait: const Duration(seconds: 1), next: const Duration(seconds: 1)); + } + final doubled = retryAfter * 2; + return ( + wait: retryAfter, + next: doubled < maxIsolateRestartDelay ? doubled : maxIsolateRestartDelay, + ); +} + +/// How long a shutdown waits for in-flight requests before closing the +/// connections that remain, from `SHUTDOWN_TIMEOUT` in seconds. +Duration get shutdownTimeout => + Duration(seconds: env('SHUTDOWN_TIMEOUT', 10)); diff --git a/packages/core/lib/src/server/uncaught_errors.dart b/packages/core/lib/src/server/uncaught_errors.dart new file mode 100644 index 0000000..7b1e052 --- /dev/null +++ b/packages/core/lib/src/server/uncaught_errors.dart @@ -0,0 +1,24 @@ +import 'dart:isolate'; + +import '../logger/logger.dart'; + +/// Keeps this isolate serving when app code leaves an error unhandled — +/// a future nobody awaited that fails later, say. +/// +/// Such an error belongs to no request, so no handler can catch it, and by +/// default it stops the isolate: every request it is serving, and in the +/// main isolate the whole process. Instead it is logged, when [log] is +/// set, and the isolate carries on. A server isolate started by another +/// passes `log: false`, since its errors already reach the log through +/// the isolate that started it. +void keepServingOnUncaughtErrors({required bool log}) { + if (log) { + final errors = ReceivePort(); + errors.listen((message) { + final [error, stack] = message as List; + Logger.log('Uncaught error: $error\n$stack', type: Logger.ERROR); + }); + Isolate.current.addErrorListener(errors.sendPort); + } + Isolate.current.setErrorsFatal(false); +} diff --git a/packages/core/lib/src/utils/helper.dart b/packages/core/lib/src/utils/helper.dart index afb6d05..e2641c6 100644 --- a/packages/core/lib/src/utils/helper.dart +++ b/packages/core/lib/src/utils/helper.dart @@ -43,3 +43,8 @@ Future deleteSession(String key) async => await IoCContainer().resolve().deleteSession(key); Future destroyAllSessions() async => await IoCContainer().resolve().destroyAllSessions(); + +/// Moves the session to a new id, keeping its data. Call it when a user +/// signs in — see [SessionManager.regenerateSession]. +Future regenerateSession() async => + await IoCContainer().resolve().regenerateSession(); diff --git a/packages/core/lib/src/utils/request_helper.dart b/packages/core/lib/src/utils/request_helper.dart index 6ec0408..b9ac903 100644 --- a/packages/core/lib/src/utils/request_helper.dart +++ b/packages/core/lib/src/utils/request_helper.dart @@ -34,7 +34,11 @@ Future requestBody() async { final req = _current; if (req == null) return {}; final m = req.method.toLowerCase(); - if (m == 'post' || m == 'patch' || m == 'put' || m == 'delete') { + if (m == 'post' || + m == 'patch' || + m == 'put' || + m == 'delete' || + m == 'query') { return await RequestBody.extractBody(request: req); } return {}; diff --git a/packages/core/pubspec.yaml b/packages/core/pubspec.yaml index 8b5eda9..9f38353 100644 --- a/packages/core/pubspec.yaml +++ b/packages/core/pubspec.yaml @@ -1,8 +1,8 @@ name: vania description: Fast, simple, and powerful backend framework for Dart built with -version: 2.0.0 +version: 2.1.0 homepage: https://vdart.dev -repository: https://github.com/vania-dart/framework/packages/core +repository: https://github.com/vania-dart/framework issue_tracker: https://github.com/vania-dart/framework/issues documentation: https://vdart.dev/docs topics: [ server, backend, httpserver, framework, web ] diff --git a/packages/core/test/fixtures/isolate_setup_app.dart b/packages/core/test/fixtures/isolate_setup_app.dart new file mode 100644 index 0000000..5640e85 --- /dev/null +++ b/packages/core/test/fixtures/isolate_setup_app.dart @@ -0,0 +1,41 @@ +import 'dart:isolate'; + +import 'package:vania/http/response.dart'; +import 'package:vania/route.dart'; +import 'package:vania/service_provider.dart'; +import 'package:vania/vania.dart'; + +/// Set by the app's setup, in whichever isolate runs it. +bool _setupRan = false; + +/// Set in `main()`, which only the first isolate runs. +bool _mainRan = false; + +class _Routes extends ServiceProvider { + @override + Future register() async { + Router.get( + '/setup', + () => Response.json({ + 'isolate': Isolate.current.debugName, + 'setup': _setupRan, + 'main': _mainRan, + }), + ); + } + + @override + Future boot() async {} +} + +/// A server for `isolate_setup_test.dart`, configured through the +/// environment it is started with. +void main() async { + _mainRan = true; + await Application().initialize( + config: { + 'providers': [_Routes()], + }, + setup: () => _setupRan = true, + ); +} diff --git a/packages/core/test/unit/content_disposition_test.dart b/packages/core/test/unit/content_disposition_test.dart new file mode 100644 index 0000000..edfa282 --- /dev/null +++ b/packages/core/test/unit/content_disposition_test.dart @@ -0,0 +1,109 @@ +import 'dart:io'; +import 'dart:typed_data'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// The name a client saves the download under, read the way browsers do: +/// `filename*` (RFC 8187, UTF-8) when present, else `filename`. +String? _savedAs(String? disposition) { + if (disposition == null) return null; + final extended = RegExp(r"filename\*=UTF-8''([^;]+)").firstMatch(disposition); + if (extended != null) return Uri.decodeComponent(extended.group(1)!); + return RegExp(r'filename="([^"]*)"').firstMatch(disposition)?.group(1); +} + +void main() { + late Directory dir; + late HttpServer server; + late HttpClient client; + final bytes = Uint8List.fromList([1, 2, 3]); + + setUpAll(() async { + dir = Directory.systemTemp.createTempSync('vania_disposition'); + final file = File('${dir.path}/file.pdf')..writeAsBytesSync(bytes); + + clearRouteCaches(); + Router.basePrefix(null); + // The name comes in the query string, which arrives decoded. + Router.get( + '/path', + (Request req) => Response.fromPath( + file.path, + download: true, + fileName: req.query('name'), + ), + ); + Router.get( + '/bytes', + (Request req) => Response.download(req.query('name'), bytes), + ); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + dir.deleteSync(recursive: true); + }); + + Future<(int, String?, int)> download(String route, String name) async { + final rq = await client.get( + '127.0.0.1', + server.port, + '/$route?name=${Uri.encodeQueryComponent(name)}', + ); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + final body = await res.fold>([], (a, b) => a..addAll(b)); + return ( + res.statusCode, + res.headers.value('content-disposition'), + body.length, + ); + } + + for (final route in ['path', 'bytes']) { + group('Response.${route == 'path' ? 'fromPath' : 'download'}', () { + test('a Persian file name is saved as it is', () async { + final (status, disposition, length) = await download( + route, + 'گزارش ماهانه.pdf', + ); + expect(status, 200); + expect(length, 3); + expect(_savedAs(disposition), 'گزارش ماهانه.pdf'); + // Clients that only read `filename` still get a usable name. + expect(disposition, contains('filename="')); + }); + + test('an ASCII name keeps its plain form', () async { + final (status, disposition, _) = await download(route, 'report.pdf'); + expect(status, 200); + expect(disposition, 'attachment; filename="report.pdf"'); + }); + + test('quotes and line breaks cannot break out of the header', () async { + final (status, disposition, _) = await download( + route, + 'a"b\r\nSet-Cookie: x=1.pdf', + ); + expect(status, 200); + expect(disposition, isNot(contains('\n'))); + expect( + RegExp(r'filename="([^"]*)"').firstMatch(disposition!), + isNotNull, + ); + }); + }); + } +} diff --git a/packages/core/test/unit/csrf_methods_test.dart b/packages/core/test/unit/csrf_methods_test.dart new file mode 100644 index 0000000..ce7044b --- /dev/null +++ b/packages/core/test/unit/csrf_methods_test.dart @@ -0,0 +1,180 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/config/config.dart'; +import 'package:vania/src/http/middleware/middleware.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_gate.dart'; +import 'package:vania/src/http/session/session_manager.dart'; +import 'package:vania/src/ioc_container.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// Counts the requests that reach it. +class _Records extends Middleware { + static int runs = 0; + + @override + Future handle(Request req) async => runs++; +} + +/// CSRF protection with `CSRF_PROTECTION_ENABLED`: every method that +/// changes state is checked, and only those. +void main() { + late Directory sandbox; + late HttpServer server; + late HttpClient client; + final sessions = SessionManager(); + + setUpAll(() async { + Env().load(file: File('test/.env')); + // Read once, on first use, so it is set before any request. + Env().env['CSRF_PROTECTION_ENABLED'] = 'true'; + resetSessionGate(); + IoCContainer().register( + SessionManager.new, + singleton: true, + ); + sandbox = Directory.systemTemp.createTempSync('vania_csrf'); + SessionFileStore() + ..sessionPath = sandbox.path + ..invalidateCache(); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/form', () => Response.json({})); + Router.any('/posts/{id}', (int id) => Response.json({'id': id})); + Router.post('/webhooks/stripe', () => Response.json({})); + Router.post('/webhooks-admin/purge', () => Response.json({})); + Router.post('/billing/hooks/paid', () => Response.json({})); + Config().setApplicationConfig = { + 'csrf_except': ['webhooks', 'billing.hooks/*'], + }; + Router.post('/recorded', () => Response.json({})).middleware([_Records()]); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + Env().env.remove('CSRF_PROTECTION_ENABLED'); + resetSessionGate(); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + /// A browser that has loaded a page: its cookies, and the token the + /// page would have embedded in its forms. + Future<(List, String)> openPage() async { + final rq = await client.get('127.0.0.1', server.port, '/form'); + rq.headers.set('accept', 'text/html'); + final res = await rq.close(); + await res.drain(); + final sessionId = res.cookies + .firstWhere((c) => c.name == sessions.sessionKey) + .value; + final stored = await SessionFileStore().retrieveSession(sessionId); + return ( + [for (final c in res.cookies) Cookie(c.name, c.value)], + stored!['x_csrf_token'] as String, + ); + } + + Future send( + String method, { + List cookies = const [], + String? token, + }) async { + final rq = await client.open(method, '127.0.0.1', server.port, '/posts/1'); + rq.headers.set('accept', 'application/json'); + rq.cookies.addAll(cookies); + if (token != null) rq.headers.set('X-CSRF-TOKEN', token); + if (method == 'QUERY') { + rq.headers.contentType = ContentType.json; + rq.write('{}'); + } + final res = await rq.close(); + await res.drain(); + return res.statusCode; + } + + const stateChanging = ['POST', 'PUT', 'PATCH', 'DELETE']; + + test('a state-changing request without a token is refused', () async { + for (final method in stateChanging) { + expect(await send(method), 419, reason: method); + } + }); + + test('a state-changing request with a wrong token is refused', () async { + final (cookies, _) = await openPage(); + for (final method in stateChanging) { + expect( + await send(method, cookies: cookies, token: 'forged'), + 419, + reason: method, + ); + } + }); + + test('a state-changing request with the page token goes through', () async { + final (cookies, token) = await openPage(); + for (final method in stateChanging) { + expect( + await send(method, cookies: cookies, token: token), + 200, + reason: method, + ); + } + }); + + test('a forged request is turned away before the app\'s own middleware ' + 'runs', () async { + _Records.runs = 0; + final rq = await client.post('127.0.0.1', server.port, '/recorded'); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + await res.drain(); + expect(res.statusCode, 419); + expect(_Records.runs, 0); + }); + + group('csrf_except', () { + Future post(String path) async { + final rq = await client.post('127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + await res.drain(); + return res.statusCode; + } + + test('a listed path, and the paths under it, are left out', () async { + expect(await post('/webhooks/stripe'), 200); + }); + + test('a path that only starts with the same letters is not', () async { + expect(await post('/webhooks-admin/purge'), 419); + }); + + test('a dot in a pattern is a dot, not any character', () async { + expect(await post('/billing/hooks/paid'), 419); + }); + }); + + test('a safe request needs no token', () async { + for (final method in ['GET', 'QUERY']) { + expect(await send(method), 200, reason: method); + } + }); +} diff --git a/packages/core/test/unit/env_typed_get_test.dart b/packages/core/test/unit/env_typed_get_test.dart new file mode 100644 index 0000000..d6a3270 --- /dev/null +++ b/packages/core/test/unit/env_typed_get_test.dart @@ -0,0 +1,33 @@ +import 'package:test/test.dart'; +import 'package:vania/env.dart'; + +/// Reading a typed value with `env(key, default)`. +void main() { + tearDown( + () => Env().env + ..remove('EMPTY') + ..remove('BLANK') + ..remove('SET'), + ); + + test('an empty value is no value: a typed read gets the default', () { + // `HTTP_BACKLOG=` in a .env file, say, with nothing after the `=`. + Env().env['EMPTY'] = ''; + Env().env['BLANK'] = ' '; + expect(env('EMPTY', 1024), 1024); + expect(env('BLANK', 7), 7); + expect(env('EMPTY', 1.5), 1.5); + expect(env('EMPTY', 2), 2); + expect(env('EMPTY', true), isTrue); + }); + + test('an empty string is still an empty string', () { + Env().env['EMPTY'] = ''; + expect(env('EMPTY', 'default'), ''); + }); + + test('a set value is read as before', () { + Env().env['SET'] = '42'; + expect(env('SET', 1), 42); + }); +} diff --git a/packages/core/test/unit/error_view_status_test.dart b/packages/core/test/unit/error_view_status_test.dart new file mode 100644 index 0000000..570470c --- /dev/null +++ b/packages/core/test/unit/error_view_status_test.dart @@ -0,0 +1,84 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/exception/internal_server_error.dart'; +import 'package:vania/src/exception/page_expired_exception.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_gate.dart'; +import 'package:vania/src/http/session/session_manager.dart'; +import 'package:vania/src/ioc_container.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// An app's own error pages, `lib/resources/view/errors/*.html`, keep the +/// status of the error they show. +void main() { + // The view engine reads templates relative to the working directory. + final views = Directory('lib/resources'); + late Directory sessions; + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + expect(views.existsSync(), isFalse, reason: 'would delete real views'); + final errors = Directory('lib/resources/view/errors') + ..createSync(recursive: true); + for (final code in ['404', '419', '500']) { + File('${errors.path}/$code.html').writeAsStringSync('page $code'); + } + + Env().load(file: File('test/.env')); + resetSessionGate(); + IoCContainer().register( + SessionManager.new, + singleton: true, + ); + sessions = Directory.systemTemp.createTempSync('vania_error_views'); + SessionFileStore() + ..sessionPath = sessions.path + ..invalidateCache(); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/broken', () => throw InternalServerError(message: 'x')); + Router.get('/expired', () => throw PageExpiredException()); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + resetSessionGate(); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sessions.deleteSync(recursive: true); + views.deleteSync(recursive: true); + }); + + Future<(int, String)> page(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.headers.set('accept', 'text/html'); + final res = await rq.close(); + return (res.statusCode, await res.transform(systemEncoding.decoder).join()); + } + + test('the 404 page is a 404', () async { + expect(await page('/nowhere'), (404, 'page 404')); + }); + + test('the 500 page is a 500', () async { + expect(await page('/broken'), (500, 'page 500')); + }); + + test('the 419 page is a 419', () async { + expect(await page('/expired'), (419, 'page 419')); + }); +} diff --git a/packages/core/test/unit/exception_html_response_test.dart b/packages/core/test/unit/exception_html_response_test.dart new file mode 100644 index 0000000..a8487de --- /dev/null +++ b/packages/core/test/unit/exception_html_response_test.dart @@ -0,0 +1,56 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/exception/base_http_exception.dart'; +import 'package:vania/src/exception/forbidden_exception.dart'; +import 'package:vania/src/exception/not_found_exception.dart'; +import 'package:vania/src/exception/throttle_exception.dart'; +import 'package:vania/src/http/response/response.dart'; + +/// Writes [response] once and returns what reaches the client. +Future<(int, String, HttpHeaders)> _send(Response response) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen((req) => response.makeResponse(req.response)); + final client = HttpClient(); + final res = await (await client.get('127.0.0.1', server.port, '/')).close(); + final body = await res.transform(systemEncoding.decoder).join(); + client.close(force: true); + await server.close(force: true); + return (res.statusCode, body, res.headers); +} + +void main() { + test('an HTML error page carries the error status', () async { + final notFound = NotFoundException(message: 'Gone'); + expect((await _send(notFound.response(true))).$1, 404); + + final forbidden = ForbiddenException(message: 'No'); + expect((await _send(forbidden.response(true))).$1, 403); + }); + + test('a map message shows its text, not the map', () async { + final error = NotFoundException(message: {'message': 'Not found'}); + final (_, body, _) = await _send(error.response(true)); + expect(body, 'Not found'); + }); + + test('the message is escaped in HTML', () async { + const error = BaseHttpResponseException( + message: '', + code: 400, + ); + final (_, body, _) = await _send(error.response(true)); + expect(body, '<script>x</script>'); + }); + + test('a throttled HTML request gets 429 and keeps Retry-After', () async { + final error = ThrottleException( + message: 'Slow down', + code: 429, + headers: {'Retry-After': '30'}, + ); + final (status, _, headers) = await _send(error.response(true)); + expect(status, 429); + expect(headers.value('retry-after'), '30'); + }); +} diff --git a/packages/core/test/unit/flash_persistence_test.dart b/packages/core/test/unit/flash_persistence_test.dart new file mode 100644 index 0000000..8d44ca8 --- /dev/null +++ b/packages/core/test/unit/flash_persistence_test.dart @@ -0,0 +1,128 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/flash_messages.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_gate.dart'; +import 'package:vania/src/http/session/session_manager.dart'; +import 'package:vania/src/ioc_container.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late Directory sandbox; + late HttpServer server; + late HttpClient client; + final sessionCookie = SessionManager().sessionKey; + + setUpAll(() async { + Env().load(file: File('test/.env')); + Env().env['SESSION_ENABLED'] = 'true'; + // Registered by Application in an app. + IoCContainer().register( + SessionManager.new, + singleton: true, + ); + resetSessionGate(); + sandbox = Directory.systemTemp.createTempSync('vania_flash'); + SessionFileStore() + ..sessionPath = sandbox.path + ..invalidateCache(); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/save', () => Response.back('status', 'Saved')); + Router.get('/flash-and-render', () { + FlashMessages().flash('status', 'Shown now'); + // What rendering a view does once the page has used the values. + FlashMessages().clear(); + return Response.json({}); + }); + Router.get('/submit', (Request req) async { + await req.validate({'email': 'required'}); + return Response.json({'ok': true}); + }); + Router.get( + '/page', + () => Response.json({ + 'messages': FlashMessages().messages, + 'errors': FlashMessages().errors, + 'old': FlashMessages().old, + }), + ); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + Env().env.remove('SESSION_ENABLED'); + resetSessionGate(); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + /// One browser: the session cookie set by a response goes back with + /// every later request. + late String? session; + setUp(() => session = null); + + Future<(int, Map?)> visit(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.followRedirects = false; + rq.headers + ..set('accept', 'text/html,application/json') + ..set('referer', 'http://127.0.0.1:${server.port}/page'); + if (session != null) rq.cookies.add(Cookie(sessionCookie, session!)); + final res = await rq.close(); + for (final c in res.cookies) { + if (c.name == sessionCookie) session = c.value; + } + final body = await res.transform(utf8.decoder).join(); + return ( + res.statusCode, + body.isEmpty ? null : jsonDecode(body) as Map, + ); + } + + test('a flash message reaches the next request, once', () async { + final (status, _) = await visit('/save'); + expect(status, 302); + + expect((await visit('/page')).$2!['messages'], {'status': 'Saved'}); + expect((await visit('/page')).$2!['messages'], isEmpty); + }); + + test('validation errors and old input survive the redirect back, ' + 'passwords left out', () async { + final (status, _) = await visit( + '/submit?name=Ann&password=secret&password_confirmation=secret', + ); + expect(status, 302); + + final page = (await visit('/page')).$2!; + expect((page['errors'] as Map).keys, ['email']); + expect(page['old'], {'name': 'Ann'}); + + final again = (await visit('/page')).$2!; + expect(again['errors'], isEmpty); + expect(again['old'], isEmpty); + }); + + test('a message already shown by this request is not carried over', () async { + await visit('/flash-and-render'); + expect((await visit('/page')).$2!['messages'], isEmpty); + }); +} diff --git a/packages/core/test/unit/head_method_test.dart b/packages/core/test/unit/head_method_test.dart new file mode 100644 index 0000000..7042a85 --- /dev/null +++ b/packages/core/test/unit/head_method_test.dart @@ -0,0 +1,98 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; +import 'package:vania/src/route/set_static_path.dart'; + +/// HEAD asks for exactly what GET would answer, minus the body (RFC 9110 +/// §9.3.2). Health checks and link checkers rely on it. +void main() { + late Directory sandbox; + late HttpServer server; + late HttpClient client; + var getRuns = 0; + + setUpAll(() async { + sandbox = Directory.systemTemp.createTempSync('vania_head'); + Directory('${sandbox.path}/public').createSync(); + File('${sandbox.path}/public/app.css').writeAsStringSync('body{}' * 10); + File('${sandbox.path}/video.mp4').writeAsBytesSync(List.filled(500, 1)); + staticPublicRoot = '${sandbox.path}/public'; + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/status', () { + getRuns++; + return Response.json({'status': 'ok'}); + }); + Router.post('/only-post', () => Response.json({})); + Router.get('/video', () => Response.fromPath('${sandbox.path}/video.mp4')); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + staticPublicRoot = 'public'; + sandbox.deleteSync(recursive: true); + }); + + Future<(HttpClientResponse, List)> send( + String method, + String path, + ) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + return (res, await res.fold>([], (a, b) => a..addAll(b))); + } + + test('HEAD on a GET route answers as GET would, without the body', () async { + final (get, getBody) = await send('GET', '/status'); + final runsAfterGet = getRuns; + final (head, headBody) = await send('HEAD', '/status'); + + expect(head.statusCode, 200); + expect(headBody, isEmpty); + expect(head.headers.contentLength, getBody.length); + expect( + head.headers.contentType?.mimeType, + get.headers.contentType?.mimeType, + ); + expect(getRuns, runsAfterGet + 1, reason: 'the GET action answers it'); + }); + + test('HEAD on a static file sends its headers, not the file', () async { + final (res, body) = await send('HEAD', '/app.css'); + expect(res.statusCode, 200); + expect(body, isEmpty); + expect(res.headers.contentLength, 60); + expect(res.headers.value('etag'), isNotNull); + }); + + test('HEAD on a file response sends its headers, not the file', () async { + final (res, body) = await send('HEAD', '/video'); + expect(res.statusCode, 200); + expect(body, isEmpty); + expect(res.headers.contentLength, 500); + expect(res.headers.value('accept-ranges'), 'bytes'); + }); + + test('HEAD on a path with no GET route is still a 404', () async { + expect((await send('HEAD', '/only-post')).$1.statusCode, 404); + expect((await send('HEAD', '/nowhere')).$1.statusCode, 404); + }); + + test('OPTIONS lists HEAD wherever GET is allowed', () async { + final (res, _) = await send('OPTIONS', '/status'); + expect(res.headers.value('allow'), 'GET, HEAD, OPTIONS'); + }); +} diff --git a/packages/core/test/unit/isolate_restart_delay_test.dart b/packages/core/test/unit/isolate_restart_delay_test.dart new file mode 100644 index 0000000..f45288f --- /dev/null +++ b/packages/core/test/unit/isolate_restart_delay_test.dart @@ -0,0 +1,28 @@ +import 'package:test/test.dart'; +import 'package:vania/src/server/server_isolates.dart'; + +void main() { + test('an isolate that keeps dying at start waits longer each time, ' + 'up to a minute', () { + var retryAfter = const Duration(seconds: 1); + final waits = []; + for (var i = 0; i < 9; i++) { + final (:wait, :next) = isolateRestartDelay( + retryAfter, + crashedEarly: true, + ); + waits.add(wait.inSeconds); + retryAfter = next; + } + expect(waits, [1, 2, 4, 8, 16, 32, 60, 60, 60]); + }); + + test('one that ran a while is replaced at once, and starts over', () { + final (:wait, :next) = isolateRestartDelay( + const Duration(seconds: 32), + crashedEarly: false, + ); + expect(wait, const Duration(seconds: 1)); + expect(next, const Duration(seconds: 1)); + }); +} diff --git a/packages/core/test/unit/isolate_setup_test.dart b/packages/core/test/unit/isolate_setup_test.dart new file mode 100644 index 0000000..45560c1 --- /dev/null +++ b/packages/core/test/unit/isolate_setup_test.dart @@ -0,0 +1,78 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; + +/// With `APP_ISOLATES`, every isolate but the first starts without running +/// `main()`; the app's `setup` is what reaches all of them. +void main() { + late Process server; + late int port; + + setUpAll(() async { + final probe = await ServerSocket.bind(InternetAddress.loopbackIPv4, 0); + port = probe.port; + await probe.close(); + + server = await Process.start( + Platform.resolvedExecutable, + ['test/fixtures/isolate_setup_app.dart'], + environment: { + 'APP_KEY': 'kQ1spu2E1r452fLa7cSihi7p1pdBR7040H8-UxtXNXg8=', + 'APP_ISOLATES': '2', + 'APP_HOST': '127.0.0.1', + 'APP_PORT': '$port', + 'APP_DEBUG': 'false', + }, + ); + server.stdout.drain(); + server.stderr.drain(); + }); + + tearDownAll(() async { + server.kill(ProcessSignal.sigterm); + await server.exitCode.timeout( + const Duration(seconds: 15), + onTimeout: () { + server.kill(ProcessSignal.sigkill); + return -1; + }, + ); + }); + + /// What one isolate reports, over a connection of its own so that the + /// requests spread over the isolates. + Future?> ask() async { + final client = HttpClient(); + try { + final res = await (await client.get('127.0.0.1', port, '/setup')).close(); + return jsonDecode(await res.transform(utf8.decoder).join()) + as Map; + } on SocketException { + return null; + } finally { + client.close(force: true); + } + } + + test('setup runs in every isolate, main() only in the first', () async { + final answers = >{}; + final waited = Stopwatch()..start(); + while (answers.length < 2 && waited.elapsed < const Duration(seconds: 60)) { + final answer = await ask(); + if (answer == null) { + // Still starting. + await Future.delayed(const Duration(milliseconds: 200)); + continue; + } + answers[answer['isolate'].toString()] = answer; + } + + expect(answers, hasLength(2), reason: 'both isolates answered'); + for (final answer in answers.values) { + expect(answer['setup'], isTrue, reason: '$answer'); + } + // What a worker misses without setup: the code in main(). + expect(answers.values.where((a) => a['main'] == true), hasLength(1)); + }, timeout: const Timeout(Duration(seconds: 90))); +} diff --git a/packages/core/test/unit/logger_test.dart b/packages/core/test/unit/logger_test.dart new file mode 100644 index 0000000..ab09419 --- /dev/null +++ b/packages/core/test/unit/logger_test.dart @@ -0,0 +1,44 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/logger/logger.dart'; +import 'package:vania/src/utils/helper.dart'; + +void main() { + const name = 'logger_test'; + final file = File('${storagePath('logs')}/$name.log'); + + setUp(() async { + await Logger.flush(); + if (file.existsSync()) file.deleteSync(); + }); + + tearDown(() async { + await Logger.flush(); + if (file.existsSync()) file.deleteSync(); + }); + + test('a burst of entries all reach the file, whole and in order', () async { + const count = 2000; + for (var i = 0; i < count; i++) { + Logger.log('entry $i', type: Logger.ERROR, fileName: name); + } + await Logger.flush(); + + final lines = file.readAsLinesSync(); + expect(lines, hasLength(count)); + for (var i = 0; i < count; i++) { + expect(lines[i], endsWith(' ERROR: entry $i')); + } + }); + + test('a log file that cannot be written does not raise an error, and ' + 'other log files keep working', () async { + // The directory does not exist, so the write fails asynchronously. + Logger.log('lost', fileName: 'no/such/dir/$name'); + Logger.log('kept', fileName: name); + await Logger.flush(); + + expect(file.readAsLinesSync().single, endsWith(' INFO: kept')); + }); +} diff --git a/packages/core/test/unit/middleware_chain_test.dart b/packages/core/test/unit/middleware_chain_test.dart index 87841e3..dcf7767 100644 --- a/packages/core/test/unit/middleware_chain_test.dart +++ b/packages/core/test/unit/middleware_chain_test.dart @@ -33,6 +33,14 @@ class _Wrapping extends Middleware { } } +class _CallsNextTwice extends Middleware { + @override + Future process(Request req, Next next) async { + await next(); + await next(); + } +} + void main() { setUp(log.clear); @@ -87,6 +95,14 @@ void main() { expect(log, equals(['controller'])); }); + test( + 'calling next() twice is an error, and the controller runs once', + () async { + await expectLater(run([_CallsNextTwice()]), throwsA(isA())); + expect(log.where((e) => e == 'controller').length, equals(1)); + }, + ); + test('each middleware runs exactly once', () async { await run([_Wrapping('w'), _Legacy('a'), _Legacy('b')]); expect(log.where((e) => e == 'a').length, equals(1)); diff --git a/packages/core/test/unit/query_method_test.dart b/packages/core/test/unit/query_method_test.dart new file mode 100644 index 0000000..db28262 --- /dev/null +++ b/packages/core/test/unit/query_method_test.dart @@ -0,0 +1,179 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// The HTTP QUERY method, RFC 10008: safe and idempotent like GET, with +/// the query carried in the request content like POST. +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/contacts', () => Response.json({'via': 'GET'})); + Router.query( + '/contacts', + (Request req) => Response.json({'via': 'QUERY', 'input': req.all()}), + ).acceptQuery(['application/json', 'application/x-www-form-urlencoded']); + Router.query( + '/sql', + (Request req) => Response.json({'sql': utf8.decode(req.rawBody)}), + ); + Router.any('/anything', (Request req) => Response.json([req.method])); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future<(HttpClientResponse, String)> send( + String method, + String path, { + String? contentType, + String body = '', + }) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + if (contentType != null) rq.headers.set('content-type', contentType); + rq.write(body); + final res = await rq.close(); + return (res, await res.transform(utf8.decoder).join()); + } + + test('a QUERY route receives its content as input', () async { + final (res, body) = await send( + 'QUERY', + '/contacts?page=2', + contentType: 'application/json', + body: '{"surname":"Smith","limit":10}', + ); + expect(res.statusCode, 200); + expect(jsonDecode(body), { + 'via': 'QUERY', + 'input': {'surname': 'Smith', 'limit': 10, 'page': '2'}, + }); + }); + + test('form-encoded content is input too', () async { + final (_, body) = await send( + 'QUERY', + '/contacts', + contentType: 'application/x-www-form-urlencoded', + body: 'email=%2A%40example.org&limit=10', + ); + expect(jsonDecode(body)['input'], { + 'email': '*@example.org', + 'limit': '10', + }); + }); + + test('GET and QUERY on the same path reach their own routes', () async { + expect((await send('GET', '/contacts')).$2, '{"via":"GET"}'); + }); + + test('content of a type the framework does not parse is there raw', () async { + final (res, body) = await send( + 'QUERY', + '/sql', + contentType: 'application/sql', + body: 'SELECT surname, email FROM contacts LIMIT 10', + ); + expect(res.statusCode, 200); + expect(jsonDecode(body), { + 'sql': 'SELECT surname, email FROM contacts LIMIT 10', + }); + }); + + group('a QUERY the server cannot read', () { + test('without a Content-Type is a 400', () async { + final (res, _) = await send('QUERY', '/sql', body: 'SELECT 1'); + expect(res.statusCode, 400); + }); + + test('whose content contradicts its Content-Type is a 400', () async { + final (res, _) = await send( + 'QUERY', + '/contacts', + contentType: 'application/json', + body: '{not json', + ); + expect(res.statusCode, 400); + }); + + test('of a media type the route does not take is a 415 that says ' + 'which it does', () async { + final (res, _) = await send( + 'QUERY', + '/contacts', + contentType: 'application/sql', + body: 'SELECT 1', + ); + expect(res.statusCode, 415); + expect( + res.headers.value('accept-query'), + 'application/json, application/x-www-form-urlencoded', + ); + }); + }); + + test('a successful QUERY advertises what the route accepts', () async { + final (res, _) = await send( + 'QUERY', + '/contacts', + contentType: 'application/json', + body: '{}', + ); + expect( + res.headers.value('accept-query'), + 'application/json, application/x-www-form-urlencoded', + ); + }); + + test('OPTIONS lists the methods of the path, QUERY included', () async { + final (res, _) = await send('OPTIONS', '/contacts'); + expect(res.statusCode, 200); + expect(res.headers.value('allow'), 'GET, HEAD, QUERY, OPTIONS'); + expect( + res.headers.value('accept-query'), + 'application/json, application/x-www-form-urlencoded', + ); + }); + + test('a POST with unreadable JSON is unchanged: empty input', () async { + Router.post('/legacy', (Request req) => Response.json(req.all())); + clearRouteCaches(); + initializeRoutes(); + final (res, body) = await send( + 'POST', + '/legacy', + contentType: 'application/json', + body: '{not json', + ); + expect(res.statusCode, 200); + expect(body, '{}'); + }); + + test('Router.any answers QUERY as well', () async { + final (_, body) = await send( + 'QUERY', + '/anything', + contentType: 'application/json', + body: '{}', + ); + expect(body, '["query"]'); + }); +} diff --git a/packages/core/test/unit/request_accessors_test.dart b/packages/core/test/unit/request_accessors_test.dart new file mode 100644 index 0000000..897cf22 --- /dev/null +++ b/packages/core/test/unit/request_accessors_test.dart @@ -0,0 +1,112 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/route/route_data.dart'; + +/// POSTs [body] and hands [inspect] the server's [Request] with its body +/// already read. +Future _withBody( + String contentType, + String body, + Future Function(Request req) inspect, +) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + final done = Completer(); + server.listen((httpReq) async { + try { + final req = Request().from( + request: httpReq, + route: RouteData(method: 'post', path: '/x', action: () {}), + ); + await req.extractBody(); + await inspect(req); + done.complete(); + } catch (e, s) { + done.completeError(e, s); + } + await httpReq.response.close(); + }); + + final client = HttpClient(); + final rq = await client.post('127.0.0.1', server.port, '/x'); + rq.headers.set('content-type', contentType); + rq.add(utf8.encode(body)); + await (await rq.close()).drain(); + client.close(force: true); + try { + await done.future; + } finally { + await server.close(force: true); + } +} + +void main() { + test('has() is true for a present non-string value', () { + return _withBody('application/json', '{"age":30,"name":""}', (r) async { + expect(r.has('age'), isTrue); + expect(r.has('name'), isFalse); + expect(r.has('missing'), isFalse); + }); + }); + + test('except() leaves the request input alone', () { + return _withBody('application/json', '{"user":"a","pass":"b"}', (r) async { + expect(r.except('pass'), {'user': 'a'}); + expect(r.except(['user', 'pass']), isEmpty); + expect(r.input('pass'), 'b'); + expect(r.all(), {'user': 'a', 'pass': 'b'}); + }); + }); + + test('headers keeps values sent on separate lines apart', () async { + // HttpClient folds repeated headers onto one line, so this is sent raw. + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + final seen = Completer(); + server.listen((httpReq) async { + seen.complete(Request().from(request: httpReq).headers['x-multi']); + await httpReq.response.close(); + }); + final socket = await Socket.connect('127.0.0.1', server.port); + socket.write( + 'GET / HTTP/1.1\r\nHost: x\r\nX-Multi: a\r\nX-Multi: b\r\n' + 'Connection: close\r\n\r\n', + ); + expect(await seen.future, 'a, b'); + socket.destroy(); + await server.close(force: true); + }); + + test('file() and files() on a text field are null, not a crash', () { + const body = + '--XyZ\r\n' + 'Content-Disposition: form-data; name="avatar"\r\n\r\n' + 'not a file\r\n' + '--XyZ--\r\n'; + return _withBody('multipart/form-data; boundary=XyZ', body, (r) async { + expect(r.file('avatar'), isNull); + expect(r.files('avatar'), isNull); + expect(r.hasFile('avatar'), isFalse); + }); + }); + + test('file() and files() still return uploads', () { + const body = + '--XyZ\r\n' + 'Content-Disposition: form-data; name="docs[]"; filename="a.txt"\r\n' + 'Content-Type: text/plain\r\n\r\n' + 'one\r\n' + '--XyZ\r\n' + 'Content-Disposition: form-data; name="docs[]"; filename="b.txt"\r\n' + 'Content-Type: text/plain\r\n\r\n' + 'two\r\n' + '--XyZ--\r\n'; + return _withBody('multipart/form-data; boundary=XyZ', body, (r) async { + expect(r.file('docs')!.filename, 'a.txt'); + expect(r.files('docs')!.map((f) => f.filename), ['a.txt', 'b.txt']); + expect(r.hasFile('docs'), isTrue); + }); + }); +} diff --git a/packages/core/test/unit/request_body_size_test.dart b/packages/core/test/unit/request_body_size_test.dart index 5a1e437..8d4960f 100644 --- a/packages/core/test/unit/request_body_size_test.dart +++ b/packages/core/test/unit/request_body_size_test.dart @@ -4,6 +4,7 @@ import 'dart:io'; import 'package:test/test.dart'; import 'package:vania/src/exception/http_exception.dart'; import 'package:vania/src/http/request/request_body.dart'; +import 'package:vania/src/http/request/request_file.dart'; /// Starts a loopback HTTP server, sends a POST with the given payload /// pattern, and returns the server-side `HttpRequest` for the parser to @@ -87,6 +88,76 @@ void main() { }, ); + // A chunked upload declares no Content-Length, so only counting the + // bytes as they stream in can enforce the cap. + List multipart({required String disposition, required int size}) => [ + ...'--XyZ\r\nContent-Disposition: form-data; $disposition\r\n\r\n' + .codeUnits, + ...List.filled(size, 65), + ...'\r\n--XyZ--\r\n'.codeUnits, + ]; + + for (final (kind, disposition) in [ + ('text field', 'name="note"'), + ('file', 'name="upload"; filename="big.bin"'), + ]) { + test( + 'when a chunked multipart $kind exceeds MAX_BODY_SIZE, aborts 413', + () async { + final req = await _postWith( + body: multipart(disposition: disposition, size: 11 * 1024 * 1024), + contentType: 'multipart/form-data; boundary=XyZ', + chunkCount: 20, + declareContentLength: false, + ); + req.response.statusCode = 413; + await expectLater( + RequestBody.extractBody(request: req), + throwsA( + isA().having((e) => e.code, 'code', 413), + ), + ); + await req.response.close(); + }, + ); + } + + test( + 'when a chunked multipart body is under the limit, parses it', + () async { + final req = await _postWith( + body: multipart(disposition: 'name="note"', size: 1024), + contentType: 'multipart/form-data; boundary=XyZ', + chunkCount: 4, + declareContentLength: false, + ); + req.response.statusCode = 200; + final parsed = await RequestBody.extractBody(request: req); + expect(parsed['note'], 'A' * 1024); + await req.response.close(); + }, + ); + + test( + 'when a chunked multipart file is under the limit, its bytes are readable', + () async { + final req = await _postWith( + body: multipart( + disposition: 'name="upload"; filename="a.bin"', + size: 300 * 1024, + ), + contentType: 'multipart/form-data; boundary=XyZ', + chunkCount: 8, + declareContentLength: false, + ); + req.response.statusCode = 200; + final parsed = await RequestBody.extractBody(request: req); + final file = parsed['upload'] as RequestFile; + expect(await file.size, 300 * 1024); + await req.response.close(); + }, + ); + test('when body under limit, returns parsed JSON map', () async { final body = '{"a":1,"b":"two"}'; final req = await _postWith( diff --git a/packages/core/test/unit/request_handler_resilience_test.dart b/packages/core/test/unit/request_handler_resilience_test.dart new file mode 100644 index 0000000..b7402bf --- /dev/null +++ b/packages/core/test/unit/request_handler_resilience_test.dart @@ -0,0 +1,57 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late HttpServer server; + late HttpClient client; + + setUp(() async { + clearRouteCaches(); + Router.basePrefix(null); + // The controller has already put headers and part of a body on the + // wire when it fails, so the error response cannot be written. + Router.get('/fails-mid-response', (Request req) async { + req.response.write('partial'); + await req.response.flush(); + throw StateError('failed after the response started'); + }); + Router.get('/ok', () => Response.json({'ok': true})); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDown(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future<(int, String)> get(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + final res = await rq.close(); + final body = await res.transform(const SystemEncoding().decoder).join(); + return (res.statusCode, body); + } + + test('a failure after the response started does not take the server ' + 'down', () async { + try { + await get('/fails-mid-response'); + } on HttpException { + // The client may see a truncated response; that is acceptable. + } + + final (status, body) = await get('/ok'); + expect(status, 200); + expect(body, '{"ok":true}'); + }); +} diff --git a/packages/core/test/unit/request_lazy_test.dart b/packages/core/test/unit/request_lazy_test.dart new file mode 100644 index 0000000..a0b7c17 --- /dev/null +++ b/packages/core/test/unit/request_lazy_test.dart @@ -0,0 +1,60 @@ +import 'dart:async'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/route/route_data.dart'; + +/// Hands [inspect] a [Request] for a POST carrying a cookie and a JSON +/// body, before its body has been read — where middleware sees it. +Future _withRequest(Future Function(Request req) inspect) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + final done = Completer(); + server.listen((httpReq) async { + try { + await inspect( + Request().from( + request: httpReq, + route: RouteData(method: 'post', path: '/x', action: () {}), + ), + ); + done.complete(); + } catch (e, s) { + done.completeError(e, s); + } + await httpReq.response.close(); + }); + + final client = HttpClient(); + final rq = await client.post('127.0.0.1', server.port, '/x?q=1'); + rq.headers + ..set('cookie', 'XSRF-TOKEN=abc=; other=2') + ..contentType = ContentType.json; + rq.write('{"name":"vania"}'); + await (await rq.close()).drain(); + client.close(force: true); + try { + await done.future; + } finally { + await server.close(force: true); + } +} + +void main() { + test('cookies are readable before the body is, as middleware needs', () { + // CsrfMiddleware checks the XSRF-TOKEN cookie before it reads the body. + return _withRequest((req) async { + expect(req.cookie('XSRF-TOKEN'), 'abc='); + expect(req.cookie('other'), 2); + }); + }); + + test('input read before the body still sees the body afterwards', () { + return _withRequest((req) async { + expect(req.input('q'), '1'); + await req.extractBody(); + expect(req.input('name'), 'vania'); + expect(req.all(), {'name': 'vania', 'q': '1'}); + }); + }); +} diff --git a/packages/core/test/unit/request_lifecycle_test.dart b/packages/core/test/unit/request_lifecycle_test.dart new file mode 100644 index 0000000..b275c57 --- /dev/null +++ b/packages/core/test/unit/request_lifecycle_test.dart @@ -0,0 +1,290 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/exception/internal_server_error.dart'; +import 'package:vania/src/exception/not_found_exception.dart'; +import 'package:vania/src/exception/throttle_exception.dart'; +import 'package:vania/src/exception/unauthenticated.dart'; +import 'package:vania/src/http/middleware/middleware.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_gate.dart'; +import 'package:vania/src/http/session/session_manager.dart'; +import 'package:vania/src/ioc_container.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// Adds a header once the rest of the chain has run — the example the +/// middleware docs give. +class _Timing extends Middleware { + @override + Future process(Request req, Next next) async { + await next(); + req.response.headers.add('X-Duration-Ms', '1'); + } +} + +/// Throws whatever [error] builds, as a guard would. +class _Throws extends Middleware { + _Throws(this.error); + final Object Function() error; + + @override + Future handle(Request req) async => throw error(); +} + +/// Validates the request before the controller, as a form request would. +class _Validates extends Middleware { + @override + Future handle(Request req) async { + await req.extractBody(); + await req.validate({'email': 'required'}); + } +} + +/// Sees the controller's error on its way out. +class _Observes extends Middleware { + static Object? seen; + + @override + Future process(Request req, Next next) async { + try { + await next(); + } catch (e) { + seen = e; + rethrow; + } + } +} + +/// Ends the chain and forgets to answer. +class _Forgets extends Middleware { + @override + Future process(Request req, Next next) async {} +} + +/// Ends the chain and answers itself. +class _AnswersItself extends Middleware { + @override + Future process(Request req, Next next) async { + req.response + ..statusCode = 403 + ..write('handled here'); + await req.response.close(); + } +} + +void main() { + late Directory sandbox; + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + Env().load(file: File('test/.env')); + Env().env['SESSION_ENABLED'] = 'true'; + resetSessionGate(); + IoCContainer().register( + SessionManager.new, + singleton: true, + ); + sandbox = Directory.systemTemp.createTempSync('vania_lifecycle'); + SessionFileStore() + ..sessionPath = sandbox.path + ..invalidateCache(); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get( + '/timed', + () => Response.json({'ok': true}), + ).middleware([_Timing()]); + // A session guard asks for the browser to be sent to its login page, + // as vania_auth does; a token guard only says who is not signed in. + Unauthenticated toLogin() => + Unauthenticated(message: '/login', responseType: ResponseType.html); + Unauthenticated tokenRejected() => + Unauthenticated(message: 'Unauthenticated'); + Router.get('/login-ctrl', () => throw toLogin()); + Router.get( + '/login-mw', + () => Response.json({}), + ).middleware([_Throws(toLogin)]); + Router.get('/token-ctrl', () => throw tokenRejected()); + Router.get( + '/token-mw', + () => Response.json({}), + ).middleware([_Throws(tokenRejected)]); + Router.get( + '/missing-ctrl', + () => throw NotFoundException(message: {'error': 'no such user'}), + ); + Router.get( + '/throttled-ctrl', + () => throw ThrottleException( + message: 'Slow down', + code: 429, + headers: {'Retry-After': '30'}, + ), + ); + Router.get('/leaky-mw', () => Response.json({})).middleware([ + _Throws(() => InternalServerError(message: 'db password is hunter2')), + ]); + Router.get( + '/validate-mw', + () => Response.json({}), + ).middleware([_Validates()]); + Router.get( + '/crash-ctrl', + () => throw StateError('boom'), + ).middleware([_Observes()]); + Router.get('/forgets', () => Response.json({})).middleware([_Forgets()]); + Router.get('/back-get', () => Response.back()); + Router.post('/back-post', () => Response.back()); + Router.get( + '/answers-itself', + () => Response.json({}), + ).middleware([_AnswersItself()]); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + Env().env.remove('SESSION_ENABLED'); + resetSessionGate(); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + Future<(HttpClientResponse, String)> get( + String path, { + bool html = false, + }) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.followRedirects = false; + rq.headers + ..set('accept', html ? 'text/html' : 'application/json') + ..set('referer', 'http://127.0.0.1:${server.port}/form'); + final res = await rq.close(); + return (res, await res.transform(utf8.decoder).join()); + } + + group('the response is written once the middleware chain is done', () { + test('so a middleware can add a header after next()', () async { + final (res, body) = await get('/timed'); + expect(res.statusCode, 200); + expect(body, '{"ok":true}'); + expect(res.headers.value('x-duration-ms'), '1'); + }); + + test('and a middleware sees the controller\'s error', () async { + _Observes.seen = null; + final (res, _) = await get('/crash-ctrl'); + expect(res.statusCode, 500); + expect(_Observes.seen, isA()); + }); + }); + + group('back() with no usable Referer', () { + Future locationOf(String method, String path) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.followRedirects = false; + rq.headers + ..set('accept', 'text/html') + ..set('referer', 'https://evil.example/phish'); + final res = await rq.close(); + await res.drain(); + return res.headers.value('location'); + } + + test('from a GET goes to /, never back to the same URL', () async { + // Back to /back-get would answer itself with a redirect, forever. + expect(await locationOf('GET', '/back-get'), '/'); + }); + + test('from a form post goes to the page of the same path', () async { + expect(await locationOf('POST', '/back-post'), '/back-post'); + }); + }); + + group('a middleware that stops the chain', () { + test('and answers keeps its answer', () async { + final (res, body) = await get('/answers-itself'); + expect(res.statusCode, 403); + expect(body, 'handled here'); + }); + + test('without answering gets a 500, not a request that hangs', () async { + final (res, _) = await get( + '/forgets', + ).timeout(const Duration(seconds: 3)); + expect(res.statusCode, 500); + }); + }); + + group('an error is answered the same wherever it was thrown', () { + test('Unauthenticated redirects a browser, from controller or ' + 'middleware', () async { + for (final path in ['/login-ctrl', '/login-mw']) { + final (res, _) = await get(path, html: true); + expect(res.statusCode, 302, reason: path); + expect(res.headers.value('location'), '/login', reason: path); + } + }); + + test('Unauthenticated is a 401 for an API client either way', () async { + for (final path in ['/login-ctrl', '/login-mw']) { + expect((await get(path)).$1.statusCode, 401, reason: path); + } + }); + + test('an Unauthenticated that asks for no redirect is a 401 even for a ' + 'browser, never a redirect to its message', () async { + for (final path in ['/token-ctrl', '/token-mw']) { + final (res, _) = await get(path, html: true); + expect(res.statusCode, 401, reason: path); + expect(res.headers.value('location'), isNull, reason: path); + } + }); + + test('an error with a map message keeps its status and body', () async { + final (res, body) = await get('/missing-ctrl'); + expect(res.statusCode, 404); + expect(jsonDecode(body), {'error': 'no such user'}); + }); + + test('a throttle error keeps its Retry-After', () async { + final (res, _) = await get('/throttled-ctrl'); + expect(res.statusCode, 429); + expect(res.headers.value('retry-after'), '30'); + }); + + test('a 5xx raised on purpose does not show its message', () async { + final (res, body) = await get('/leaky-mw'); + expect(res.statusCode, 500); + expect(body, isNot(contains('hunter2'))); + }); + + test('a failed validation in middleware sends a browser back, and an ' + 'API client a 422', () async { + final (page, _) = await get('/validate-mw', html: true); + expect(page.statusCode, 302); + expect(page.headers.value('location'), '/form'); + + final (api, body) = await get('/validate-mw'); + expect(api.statusCode, 422); + expect(jsonDecode(body), contains('email')); + }); + }); +} diff --git a/packages/core/test/unit/request_timeout_abandon_test.dart b/packages/core/test/unit/request_timeout_abandon_test.dart new file mode 100644 index 0000000..4d96d4b --- /dev/null +++ b/packages/core/test/unit/request_timeout_abandon_test.dart @@ -0,0 +1,84 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_gate.dart'; +import 'package:vania/src/http/session/session_manager.dart'; +import 'package:vania/src/ioc_container.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// Once REQUEST_TIMEOUT has answered a request, the handler that is still +/// running cannot be stopped — but nothing it does later reaches the +/// client or the session. +void main() { + late Directory sessions; + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + Env().load(file: File('test/.env')); + // Read once, on first use, so they are set before any request. + Env().env['REQUEST_TIMEOUT'] = '1'; + Env().env['SESSION_ENABLED'] = 'true'; + resetSessionGate(); + IoCContainer().register( + SessionManager.new, + singleton: true, + ); + sessions = Directory.systemTemp.createTempSync('vania_timeout_abandon'); + SessionFileStore() + ..sessionPath = sessions.path + ..invalidateCache(); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/slow', () async { + await IoCContainer().resolve().setSession('late', true); + await Future.delayed(const Duration(milliseconds: 1800)); + return Response.json({'late': true}); + }); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + resetSessionGate(); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sessions.deleteSync(recursive: true); + }); + + test( + 'the handler\'s late response and session changes are dropped', + () async { + // An existing session, so it is the one the request works with. + await SessionFileStore().storeSession('sid', {'user': 1}); + + final rq = await client.get('127.0.0.1', server.port, '/slow'); + rq.headers.set('accept', 'text/html'); + rq.cookies.add(Cookie(SessionManager().sessionKey, 'sid')); + final res = await rq.close(); + await res.drain(); + expect(res.statusCode, 503); + + // Let the handler finish what it was doing. + await Future.delayed(const Duration(milliseconds: 1500)); + + SessionFileStore().invalidateCache(); + final stored = await SessionFileStore().retrieveSession('sid'); + expect(stored?['user'], 1); + expect(stored?['late'], isNull, reason: 'saved after the 503 went out'); + }, + ); +} diff --git a/packages/core/test/unit/request_timeout_test.dart b/packages/core/test/unit/request_timeout_test.dart new file mode 100644 index 0000000..55e900b --- /dev/null +++ b/packages/core/test/unit/request_timeout_test.dart @@ -0,0 +1,71 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late HttpServer server; + late HttpClient client; + + // Read once, on the first request, so it is set before any. + Env().env['REQUEST_TIMEOUT'] = '1'; + + setUp(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/hangs', () async { + await Future.delayed(const Duration(seconds: 5)); + return Response.json({'late': true}); + }); + Router.get('/quick', () => Response.json({'ok': true})); + // Answers at once, then streams for longer than REQUEST_TIMEOUT. + Router.get( + '/stream', + () => Response.sse( + Stream.periodic(const Duration(milliseconds: 400), (i) => i).take(5), + ), + ); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDown(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future<(int, String)> get(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + return (res.statusCode, await res.transform(systemEncoding.decoder).join()); + } + + test('a request running past REQUEST_TIMEOUT gets a 503', () async { + final waited = Stopwatch()..start(); + final (status, body) = await get('/hangs'); + expect(status, 503); + expect(body, '{"message":"Request timed out"}'); + expect(waited.elapsed, lessThan(const Duration(seconds: 3))); + }); + + test('a response already streaming is not cut off', () async { + final rq = await client.get('127.0.0.1', server.port, '/stream'); + final res = await rq.close(); + final body = await res.transform(systemEncoding.decoder).join(); + expect(res.statusCode, 200); + expect('data:'.allMatches(body).length, 5); + }); + + test('a request inside REQUEST_TIMEOUT is untouched', () async { + expect(await get('/quick'), (200, '{"ok":true}')); + }); +} diff --git a/packages/core/test/unit/response_correctness_test.dart b/packages/core/test/unit/response_correctness_test.dart new file mode 100644 index 0000000..507db4f --- /dev/null +++ b/packages/core/test/unit/response_correctness_test.dart @@ -0,0 +1,112 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/empty', () => Response()); + Router.get('/back', () => Response.back()); + Router.get('/slug/{s}', (String s) => Response.json([s])).whereString('s'); + Router.get('/num/{x}', (double x) => Response.json([x])).whereDouble('x'); + Router.get('/flag/{b}', (bool b) => Response.json([b])).whereBool('b'); + Router.get('/count/{n}', (int n) => Response.json([n])).whereInt('n'); + Router.get('/validate', (Request req) async { + await req.validate({'name': 'required'}); + return Response.json({'ok': true}); + }); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future<(HttpClientResponse, String)> get( + String path, { + Map headers = const {}, + }) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.followRedirects = false; + headers.forEach(rq.headers.set); + final res = await rq.close(); + return (res, await res.transform(systemEncoding.decoder).join()); + } + + test('a response with no data has an empty body, not "null"', () async { + final (res, body) = await get('/empty'); + expect(res.statusCode, 200); + expect(body, isEmpty); + }); + + group('Response.back()', () { + test('goes to a same-origin Referer', () async { + final (res, _) = await get( + '/back', + headers: {'referer': 'http://127.0.0.1:${server.port}/form?step=2'}, + ); + expect(res.statusCode, 302); + expect(res.headers.value('location'), '/form?step=2'); + }); + + test('ignores a Referer from another site', () async { + final (res, _) = await get( + '/back', + headers: {'referer': 'https://evil.example/phish'}, + ); + expect(res.headers.value('location'), '/'); + }); + + test('with nothing to go back to, goes to /', () async { + final (res, _) = await get('/back'); + expect(res.headers.value('location'), '/'); + }); + }); + + group('declared parameter types reach the controller as declared', () { + test('whereString keeps digits a String', () async { + expect((await get('/slug/2024')).$2, '["2024"]'); + }); + + test('whereDouble passes a double, and rejects a non-number', () async { + expect((await get('/num/5')).$2, '[5.0]'); + expect((await get('/num/abc')).$1.statusCode, 404); + }); + + test('whereBool passes a bool, and rejects anything else', () async { + expect((await get('/flag/TRUE')).$2, '[true]'); + expect((await get('/flag/yes')).$1.statusCode, 404); + }); + + test('whereInt is unchanged', () async { + expect((await get('/count/7')).$2, '[7]'); + expect((await get('/count/x')).$1.statusCode, 404); + }); + }); + + test('a failed validation without sessions answers 422, not a redirect ' + 'nowhere', () async { + // The app has no views, so there is no session to carry the errors + // across a redirect — even for a browser asking for HTML. + final (res, _) = await get( + '/validate', + headers: {'accept': 'text/html,application/json'}, + ); + expect(res.statusCode, 422); + }); +} diff --git a/packages/core/test/unit/response_from_path_test.dart b/packages/core/test/unit/response_from_path_test.dart new file mode 100644 index 0000000..54169b2 --- /dev/null +++ b/packages/core/test/unit/response_from_path_test.dart @@ -0,0 +1,142 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late Directory dir; + late File file; + late HttpServer server; + late HttpClient client; + final content = List.generate(1000, (i) => i % 251); + + setUpAll(() async { + dir = Directory.systemTemp.createTempSync('vania_from_path'); + file = File('${dir.path}/clip.mp4')..writeAsBytesSync(content); + + clearRouteCaches(); + Router.basePrefix(null); + Router.get('/inline', () => Response.fromPath(file.path)); + Router.get( + '/download', + () => Response.fromPath(file.path, download: true, fileName: 'a.mp4'), + ); + Router.get('/missing', () => Response.fromPath('${dir.path}/nope')); + Router.query('/inline', () => Response.fromPath(file.path)); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + dir.deleteSync(recursive: true); + }); + + Future<(HttpClientResponse, List)> get( + String path, [ + Map headers = const {}, + ]) async { + final rq = await client.get('127.0.0.1', server.port, path); + headers.forEach(rq.headers.set); + final res = await rq.close(); + final body = await res.fold>([], (a, b) => a..addAll(b)); + return (res, body); + } + + test('the whole file, advertising range support', () async { + final (res, body) = await get('/inline'); + expect(res.statusCode, 200); + expect(res.headers.contentLength, 1000); + expect(res.headers.value('accept-ranges'), 'bytes'); + expect(res.headers.contentType?.mimeType, 'video/mp4'); + expect(body, content); + }); + + test('a byte range', () async { + final (res, body) = await get('/inline', {'range': 'bytes=10-19'}); + expect(res.statusCode, 206); + expect(res.headers.value('content-range'), 'bytes 10-19/1000'); + expect(body, content.sublist(10, 20)); + }); + + test('an open-ended range and a suffix range', () async { + final (open, openBody) = await get('/inline', {'range': 'bytes=990-'}); + expect(open.headers.value('content-range'), 'bytes 990-999/1000'); + expect(openBody, content.sublist(990)); + + final (suffix, suffixBody) = await get('/inline', {'range': 'bytes=-5'}); + expect(suffix.headers.value('content-range'), 'bytes 995-999/1000'); + expect(suffixBody, content.sublist(995)); + }); + + test('a range ending past the file is cut to the file', () async { + final (res, body) = await get('/inline', {'range': 'bytes=998-5000'}); + expect(res.headers.value('content-range'), 'bytes 998-999/1000'); + expect(body, content.sublist(998)); + }); + + test('a range starting past the file is a 416', () async { + final (res, body) = await get('/inline', {'range': 'bytes=1000-1010'}); + expect(res.statusCode, 416); + expect(res.headers.value('content-range'), 'bytes */1000'); + expect(body, isEmpty); + }); + + test('several ranges, or a malformed one, get the whole file', () async { + for (final range in ['bytes=0-1,5-6', 'bytes=9-3', 'items=0-1']) { + final (res, body) = await get('/inline', {'range': range}); + expect(res.statusCode, 200, reason: range); + expect(body, content, reason: range); + } + }); + + test('If-Range for another version gets the whole file', () async { + final (res, body) = await get('/inline', { + 'range': 'bytes=0-9', + 'if-range': 'Thu, 01 Jan 1970 00:00:00 GMT', + }); + expect(res.statusCode, 200); + expect(body, content); + + final current = (await get('/inline')).$1.headers.value('last-modified')!; + final (match, _) = await get('/inline', { + 'range': 'bytes=0-9', + 'if-range': current, + }); + expect(match.statusCode, 206); + }); + + test('a download is named for saving', () async { + final (res, body) = await get('/download'); + expect( + res.headers.value('content-disposition'), + 'attachment; filename="a.mp4"', + ); + expect(body, content); + }); + + test('a QUERY takes a range as a GET does (RFC 10008 §2.8)', () async { + final rq = await client.open('QUERY', '127.0.0.1', server.port, '/inline'); + rq.headers + ..set('content-type', 'application/json') + ..set('range', 'bytes=0-9'); + rq.write('{}'); + final res = await rq.close(); + final body = await res.fold>([], (a, b) => a..addAll(b)); + expect(res.statusCode, 206); + expect(body, content.sublist(0, 10)); + }); + + test('a missing file is a 404', () async { + final (res, _) = await get('/missing'); + expect(res.statusCode, 404); + }); +} diff --git a/packages/core/test/unit/response_json_encoding_test.dart b/packages/core/test/unit/response_json_encoding_test.dart index 3fdb19c..1fa53fd 100644 --- a/packages/core/test/unit/response_json_encoding_test.dart +++ b/packages/core/test/unit/response_json_encoding_test.dart @@ -75,12 +75,14 @@ void main() { }); }); - test('an unencodable value still produces a response instead of hanging', - () async { - // DateTime has no toJson; both the old and new encoder throw on it, and - // the handler is expected to answer rather than leave the socket open. - final res = await _send(() => Response.json({'t': DateTime(2020)})); - expect(res.status, equals(HttpStatus.ok)); - expect(utf8.decode(res.body), contains('jsonEncode Error')); - }); + test( + 'an unencodable value still produces a response instead of hanging', + () async { + // DateTime has no toJson; both the old and new encoder throw on it, and + // the handler is expected to answer rather than leave the socket open. + final res = await _send(() => Response.json({'t': DateTime(2020)})); + expect(res.status, equals(HttpStatus.internalServerError)); + expect(utf8.decode(res.body), '{"message":"Internal Server Error"}'); + }, + ); } diff --git a/packages/core/test/unit/response_sse_test.dart b/packages/core/test/unit/response_sse_test.dart new file mode 100644 index 0000000..24832af --- /dev/null +++ b/packages/core/test/unit/response_sse_test.dart @@ -0,0 +1,100 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/response/response.dart'; + +/// Serves `Response.sse(events())` to every request. +Future _serve(Stream Function() events) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen((req) => Response.sse(events()).makeResponse(req.response)); + return server; +} + +/// Opens a raw connection and returns everything the server sends on it. +Future<(Socket, Stream)> _open(HttpServer server) async { + final socket = await Socket.connect('127.0.0.1', server.port); + socket.write('GET / HTTP/1.1\r\nHost: x\r\n\r\n'); + return ( + socket, + socket.cast>().transform(utf8.decoder).asBroadcastStream(), + ); +} + +void main() { + // Read once, on first use, so it is set before any stream starts. + Env().env['SSE_HEARTBEAT'] = '1'; + + late HttpServer server; + tearDown(() => server.close(force: true)); + + test('sends each event, then ends with the stream', () async { + server = await _serve( + () => Stream.fromIterable([ + 1, + 'two', + {'n': 3}, + ]), + ); + final client = HttpClient(); + final res = await (await client.get('127.0.0.1', server.port, '/')).close(); + expect(res.headers.contentType?.mimeType, 'text/event-stream'); + expect( + await res.transform(utf8.decoder).join(), + 'data: 1\n\ndata: "two"\n\ndata: {"n":3}\n\n', + ); + client.close(force: true); + }); + + test('an event reaches the client as it happens, not when the stream ' + 'ends', () async { + final source = StreamController(); + server = await _serve(() => source.stream); + final (socket, received) = await _open(server); + source.add(42); + await received + .firstWhere((s) => s.contains('data: 42')) + .timeout(const Duration(seconds: 5)); + socket.destroy(); + await source.close(); + }); + + test('a client that leaves releases the event source, even when the ' + 'source is idle', () async { + final cancelled = Completer(); + server = await _serve( + () => StreamController(onCancel: cancelled.complete).stream, + ); + + final (socket, received) = await _open(server); + await received.firstWhere((s) => s.contains('\r\n\r\n')); + socket.destroy(); + + await cancelled.future.timeout(const Duration(seconds: 5)); + }); + + test('an idle stream sends a heartbeat comment', () async { + server = await _serve(() => StreamController().stream); + final (socket, received) = await _open(server); + await received + .firstWhere((s) => s.contains(':\n\n')) + .timeout(const Duration(seconds: 5)); + socket.destroy(); + }); + + test('a failing event source ends the stream without taking the server ' + 'down', () async { + server = await _serve( + () => Stream.multi((c) { + c.add(1); + c.addError(StateError('source failed')); + }), + ); + final client = HttpClient(); + final res = await (await client.get('127.0.0.1', server.port, '/')).close(); + expect(await res.transform(utf8.decoder).join(), 'data: 1\n\n'); + client.close(force: true); + }); +} diff --git a/packages/core/test/unit/route_decode_test.dart b/packages/core/test/unit/route_decode_test.dart new file mode 100644 index 0000000..b25d318 --- /dev/null +++ b/packages/core/test/unit/route_decode_test.dart @@ -0,0 +1,88 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// A path arrives percent-encoded; routes and their parameters are text. +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.get( + '/posts/{slug}', + (String slug) => Response.json(slug), + ).whereString('slug'); + Router.get('/مقالات/{id}', (int id) => Response.json(id)).whereInt('id'); + Router.get('/درباره-ما', () => Response.json('about')); + Router.get( + '/files/{name}', + (String name) => Response.json(name), + ).whereString('name'); + Router.get('/files/a/b.txt', () => Response.json('static')); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + /// Sends [rawPath] exactly as written, percent-encoding and all. + Future<(int, Object?)> get(String rawPath) async { + final socket = await Socket.connect('127.0.0.1', server.port); + socket.write( + 'GET $rawPath HTTP/1.1\r\nHost: x\r\nAccept: application/json\r\n' + 'Connection: close\r\n\r\n', + ); + final raw = utf8.decode(await socket.expand((c) => c).toList()); + socket.destroy(); + final status = int.parse(raw.split(' ')[1]); + final body = raw.substring(raw.indexOf('\r\n\r\n') + 4); + return (status, body.isEmpty ? null : jsonDecode(body)); + } + + String encode(String path) => + path.split('/').map(Uri.encodeComponent).join('/'); + + test('a parameter reaches the controller decoded', () async { + expect(await get('/posts/${Uri.encodeComponent('سلام دنیا')}'), ( + 200, + 'سلام دنیا', + )); + expect(await get('/posts/hello%20world'), (200, 'hello world')); + }); + + test('a route written in Persian matches its encoded request', () async { + expect(await get(encode('/مقالات/7')), (200, 7)); + expect(await get(encode('/درباره-ما')), (200, 'about')); + }); + + test('an encoded slash stays inside its parameter', () async { + expect(await get('/files/a%2Fb.txt'), (200, 'a/b.txt')); + }); + + test('and does not reach the static route it would spell out', () async { + expect(await get('/files/a/b.txt'), (200, 'static')); + expect(await get('/files/a%2Fb.txt'), (200, 'a/b.txt')); + }); + + test('a plus sign in a path is a plus sign', () async { + expect(await get('/posts/c++'), (200, 'c++')); + }); + + test('a malformed escape is a 404, not a 500', () async { + expect((await get('/posts/%E0%A4%A')).$1, 404); + }); +} diff --git a/packages/core/test/unit/route_index_test.dart b/packages/core/test/unit/route_index_test.dart new file mode 100644 index 0000000..55a2f6e --- /dev/null +++ b/packages/core/test/unit/route_index_test.dart @@ -0,0 +1,165 @@ +import 'dart:async'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_data.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +Future _match(String path) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + final found = Completer(); + server.listen((req) async { + found.complete(matchRoute(req)); + await req.response.close(); + }); + final client = HttpClient(); + await (await (await client.get( + '127.0.0.1', + server.port, + path, + )).close()).drain(); + client.close(force: true); + await server.close(force: true); + return found.future; +} + +void main() { + setUp(() { + clearRouteCaches(); + Router.basePrefix(null); + }); + tearDown(clearRouteCaches); + + // Router keeps every route registered in this file, so each test uses + // its own last segment to stay out of the others' way. + group('dynamic route lookup', () { + test('when two routes match, the one registered first wins', () async { + Router.get('/{type}/{id}/a1', () {}); + Router.get('/users/{id}/a1', () {}); + initializeRoutes(); + + expect((await _match('/users/5/a1'))!.path, '/{type}/{id}/a1'); + }); + + test('in the other order, the literal route wins', () async { + Router.get('/users/{id}/b2', () {}); + Router.get('/{type}/{id}/b2', () {}); + initializeRoutes(); + + final route = await _match('/users/5/b2'); + expect(route!.path, '/users/{id}/b2'); + expect(route.params, {'id': '5'}); + }); + + test( + 'a first segment no route names falls to the parameter routes', + () async { + initializeRoutes(); + + final route = await _match('/posts/3/b2'); + expect(route!.path, '/{type}/{id}/b2'); + expect(route.params, {'type': 'posts', 'id': '3'}); + }, + ); + + test('no match is null, not an exception', () async { + Router.get('/users/{id}/c3', () {}); + initializeRoutes(); + + expect(await _match('/posts/3/c3'), isNull); + expect(await _match('/users/3/c3/extra'), isNull); + }); + + test('a match does not carry params into a later match', () async { + Router.get('/users/{id}/d4', () {}); + initializeRoutes(); + + final first = await _match('/users/1/d4'); + first!.params!['id'] = 'tampered'; + expect((await _match('/users/1/d4'))!.params, {'id': '1'}); + }); + }); + + group('unmatched requests', () { + late HttpServer server; + late HttpClient client; + + setUp(() async { + Router.get('/known', () => Response.json({'ok': true})); + initializeRoutes(); + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDown(() async { + client.close(force: true); + await server.close(force: true); + }); + + Future<(int, String)> send(String method, String path) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + return ( + res.statusCode, + await res.transform(systemEncoding.decoder).join(), + ); + } + + test('get a JSON 404', () async { + expect(await send('GET', '/nope'), (404, '{"message":"Not found"}')); + }); + + test('OPTIONS gets an empty 200, as a CORS preflight needs', () async { + expect(await send('OPTIONS', '/nope'), (200, '')); + }); + }); + + group('controller arguments', () { + late HttpServer server; + late HttpClient client; + + setUp(() async { + Router.get('/args0', () => Response.json([])); + Router.get('/args1/{a}', (int a) => Response.json([a])); + Router.get( + '/args3/{a}/{b}', + (Request req, int a, String b) => Response.json([req.path, a, b]), + ); + Router.get( + '/args5/{a}/{b}/{c}/{d}/{e}', + (int a, int b, int c, int d, int e) => Response.json([a, b, c, d, e]), + ); + initializeRoutes(); + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDown(() async { + client.close(force: true); + await server.close(force: true); + }); + + Future get(String path) async { + final res = await (await client.get( + '127.0.0.1', + server.port, + path, + )).close(); + return res.transform(systemEncoding.decoder).join(); + } + + test('reach the controller in order, whatever their number', () async { + expect(await get('/args0'), '[]'); + expect(await get('/args1/7'), '[7]'); + expect(await get('/args3/7/x'), '["/args3/{a}/{b}",7,"x"]'); + expect(await get('/args5/1/2/3/4/5'), '[1,2,3,4,5]'); + }); + }); +} diff --git a/packages/core/test/unit/route_matcher_test.dart b/packages/core/test/unit/route_matcher_test.dart index 17c693a..987b98e 100644 --- a/packages/core/test/unit/route_matcher_test.dart +++ b/packages/core/test/unit/route_matcher_test.dart @@ -115,5 +115,54 @@ void main() { ); expect(other, isNull); }); + + for (final (order, domainFirst) in [ + ('the domain route first', true), + ('the route for any host first', false), + ]) { + test('one path on a domain and for any host keeps both routes, ' + 'registered with $order', () async { + Router.basePrefix(null); + final path = '/home-${domainFirst ? 'd' : 'a'}'; + if (domainFirst) { + Router.get(path, () {}).domain('admin.site.com'); + Router.get(path, () {}); + } else { + Router.get(path, () {}); + Router.get(path, () {}).domain('admin.site.com'); + } + initializeRoutes(); + + final admin = await _resolve( + method: 'GET', + path: path, + host: 'admin.site.com', + ); + expect(admin?.domain, 'admin.site.com'); + + final other = await _resolve( + method: 'GET', + path: path, + host: 'www.site.com', + ); + expect(other, isNotNull); + expect(other!.domain, isNull); + }); + } + + test( + 'a path registered twice for the same host keeps the later route', + () async { + Router.basePrefix(null); + void first() {} + void second() {} + Router.get('/twice', first); + Router.get('/twice', second); + initializeRoutes(); + + final route = await _resolve(method: 'GET', path: '/twice'); + expect(route?.action, same(second)); + }, + ); }); } diff --git a/packages/core/test/unit/route_regex_test.dart b/packages/core/test/unit/route_regex_test.dart new file mode 100644 index 0000000..a5dd82a --- /dev/null +++ b/packages/core/test/unit/route_regex_test.dart @@ -0,0 +1,85 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +class _Items { + Response index() => Response.json('index'); + Response create() => Response.json('create'); + Response store() => Response.json('store'); + Response show(dynamic id) => Response.json('show $id'); + Response edit(dynamic id) => Response.json('edit $id'); + Response update(dynamic id) => Response.json('update $id'); + Response destroy(dynamic id) => Response.json('destroy $id'); +} + +/// A `where` pattern has to match the whole segment, as the routing docs +/// promise: "reject non-matching requests". +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.get( + '/num/{id}', + (dynamic id) => Response.json(id), + ).where('id', r'\d+'); + Router.get( + '/slug/{slug}', + (dynamic slug) => Response.json(slug), + ).where('slug', r'[a-z0-9-]+'); + Router.get( + '/anchored/{code}', + (dynamic code) => Response.json(code), + ).where('code', r'^[A-Z]{3}$'); + Router.resource('/items', _Items()); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future get(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + await res.drain(); + return res.statusCode; + } + + test('a value that only contains a match is not a match', () async { + expect(await get('/num/123'), 200); + expect(await get('/num/abc123'), 404); + expect(await get('/num/123abc'), 404); + }); + + test('a character class has to cover the whole segment', () async { + expect(await get('/slug/hello-world-2'), 200); + expect(await get('/slug/Hello!'), 404); + }); + + test('a pattern that anchors itself still works', () async { + expect(await get('/anchored/ABC'), 200); + expect(await get('/anchored/ABCD'), 404); + }); + + test('a resource id takes digits, with an optional decimal part, and ' + 'nothing else', () async { + expect(await get('/items/12'), 200); + expect(await get('/items/12.5'), 200); + expect(await get('/items/12a5'), 404); + expect(await get('/items/x12'), 404); + }); +} diff --git a/packages/core/test/unit/router_any_test.dart b/packages/core/test/unit/router_any_test.dart new file mode 100644 index 0000000..94747c7 --- /dev/null +++ b/packages/core/test/unit/router_any_test.dart @@ -0,0 +1,93 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/enum/http_request_method.dart'; +import 'package:vania/src/exception/unauthenticated.dart'; +import 'package:vania/src/http/middleware/middleware.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +class _DenyAll extends Middleware { + @override + Future handle(Request req) async => + throw Unauthenticated(message: 'denied'); +} + +/// `Router.any` registers one route per method; whatever is chained after +/// it must apply to every one of them, not only the last. +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.any( + '/guarded', + () => Response.json({'secret': 42}), + ).middleware([_DenyAll()]); + Router.any( + '/items/{id}', + (int id) => Response.json({'id': id}), + ).whereInt('id').name('items.any'); + Router.group(() { + Router.any('/panel', () => Response.json({'panel': true})); + }, prefix: 'admin'); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future send(String method, String path) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + if (method == 'QUERY') { + rq.headers.contentType = ContentType.json; + rq.write('{}'); + } + final res = await rq.close(); + await res.drain(); + return res.statusCode; + } + + const methods = ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'QUERY']; + + test('middleware chained after any guards every method', () async { + for (final method in methods) { + expect(await send(method, '/guarded'), 401, reason: method); + } + }); + + test('a constraint chained after any applies to every method', () async { + for (final method in methods) { + expect(await send(method, '/items/7'), 200, reason: method); + expect(await send(method, '/items/x'), 404, reason: method); + } + }); + + test('a name chained after any names every method', () { + final named = Router().routes.where((r) => r.name == 'items.any'); + expect( + named.map((r) => r.method).toSet(), + hasLength(HttpRequestMethod.values.length), + ); + }); + + test('any inside a group takes the group prefix', () async { + for (final method in methods) { + expect(await send(method, '/admin/panel'), 200, reason: method); + expect(await send(method, '/panel'), 404, reason: method); + } + }); +} diff --git a/packages/core/test/unit/router_resource_group_test.dart b/packages/core/test/unit/router_resource_group_test.dart new file mode 100644 index 0000000..fd7ff94 --- /dev/null +++ b/packages/core/test/unit/router_resource_group_test.dart @@ -0,0 +1,99 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/middleware/middleware.dart'; +import 'package:vania/src/http/request/request.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +/// A controller for an API: no `create` or `edit`, which only serve forms. +class _ApiController { + Response index() => Response.json('index'); + Response store() => Response.json('store'); + Response show(int id) => Response.json('show $id'); + Response update(int id) => Response.json('update $id'); + Response destroy(int id) => Response.json('destroy $id'); +} + +class _Tagged extends Middleware { + @override + Future handle(Request req) async => + req.response.headers.set('x-tagged', 'yes'); +} + +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + Router.apiResource('/photos', _ApiController(), middleware: [_Tagged()]); + + // A group whose callback fails part-way. + try { + Router.group( + () { + Router.get('/inside', () => Response.json('inside')); + throw StateError('a route file failed'); + }, + prefix: 'admin', + middleware: [_Tagged()], + ); + } on StateError { + // The app may carry on registering other route files. + } + Router.get('/outside', () => Response.json('outside')); + initializeRoutes(); + + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + Future<(int, String, String?)> send(String method, String path) async { + final rq = await client.open(method, '127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + return ( + res.statusCode, + await res.transform(systemEncoding.decoder).join(), + res.headers.value('x-tagged'), + ); + } + + group('apiResource', () { + test( + 'registers the five API actions, update under PUT and PATCH', + () async { + expect(await send('GET', '/photos'), (200, '"index"', 'yes')); + expect(await send('POST', '/photos'), (200, '"store"', 'yes')); + expect(await send('GET', '/photos/3'), (200, '"show 3"', 'yes')); + expect(await send('PUT', '/photos/3'), (200, '"update 3"', 'yes')); + expect(await send('PATCH', '/photos/3'), (200, '"update 3"', 'yes')); + expect(await send('DELETE', '/photos/3'), (200, '"destroy 3"', 'yes')); + }, + ); + + test('has no form pages', () async { + expect((await send('GET', '/photos/create')).$1, 404); + expect((await send('GET', '/photos/3/edit')).$1, 404); + }); + }); + + test( + 'a group that fails part-way does not leak its prefix or middleware', + () async { + expect(await send('GET', '/outside'), (200, '"outside"', null)); + expect((await send('GET', '/admin/outside')).$1, 404); + }, + ); +} diff --git a/packages/core/test/unit/server_shutdown_test.dart b/packages/core/test/unit/server_shutdown_test.dart new file mode 100644 index 0000000..6aedd7b --- /dev/null +++ b/packages/core/test/unit/server_shutdown_test.dart @@ -0,0 +1,66 @@ +import 'dart:async'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/server/base_http_server.dart'; + +/// A server that answers `/slow` after [slowFor], everything else at once. +Future _serve(Duration slowFor) async { + final server = BaseHttpServer(config: const {}); + final http = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.httpServer = http; + http.listen((req) async { + if (req.uri.path == '/slow') await Future.delayed(slowFor); + req.response.write('done'); + await req.response.close(); + }); + return server; +} + +Future _get(int port, String path) async { + final client = HttpClient(); + try { + final res = await (await client.get('127.0.0.1', port, path)).close(); + return '${res.statusCode} ${await res.transform(systemEncoding.decoder).join()}'; + } finally { + client.close(force: true); + } +} + +void main() { + test('an in-flight request finishes before shutdown completes, and new ' + 'connections are refused', () async { + final server = await _serve(const Duration(milliseconds: 500)); + final port = server.httpServer!.port; + + final inFlight = _get(port, '/slow'); + await Future.delayed(const Duration(milliseconds: 100)); + + final shutdown = server.shutdown(timeout: const Duration(seconds: 5)); + await Future.delayed(const Duration(milliseconds: 50)); + await expectLater( + Socket.connect('127.0.0.1', port), + throwsA(isA()), + ); + + expect(await inFlight, '200 done'); + await shutdown; + expect(server.httpServer!.connectionsInfo().total, 0); + }); + + test('a request still running at the timeout is cut off', () async { + final server = await _serve(const Duration(seconds: 30)); + final port = server.httpServer!.port; + + final stuck = _get( + port, + '/slow', + ).then((v) => v, onError: (e) => e); + await Future.delayed(const Duration(milliseconds: 100)); + + final waited = Stopwatch()..start(); + await server.shutdown(timeout: const Duration(milliseconds: 300)); + expect(waited.elapsed, lessThan(const Duration(seconds: 2))); + expect(await stuck, isA()); + }); +} diff --git a/packages/core/test/unit/session_atomic_write_test.dart b/packages/core/test/unit/session_atomic_write_test.dart new file mode 100644 index 0000000..36039b7 --- /dev/null +++ b/packages/core/test/unit/session_atomic_write_test.dart @@ -0,0 +1,72 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; + +/// With several server isolates there is no session cache, so every read +/// goes to the file — while another request may be writing it. +void main() { + late Directory sandbox; + late final SessionFileStore store; + + setUpAll(() { + Env().load(file: File('test/.env')); + // Read once, on first use: what turns the cache off. + Env().env['APP_ISOLATES'] = '2'; + store = SessionFileStore(); + }); + + setUp(() { + sandbox = Directory.systemTemp.createTempSync('vania_session_atomic'); + store + ..sessionPath = sandbox.path + ..invalidateCache(); + }); + + tearDown(() { + store + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + test('a read during a write sees the whole session, old or new', () async { + final pad = 'x' * 50000; + await store.storeSession('sid', {'version': 0, 'pad': pad}); + + final problems = []; + for (var version = 1; version <= 100; version++) { + final write = store.storeSession('sid', {'version': version, 'pad': pad}); + final reads = [ + for (var i = 0; i < 4; i++) + store + .retrieveSession('sid') + .then( + (data) => data?['pad'] == pad ? null : 'got $data', + onError: (Object e) => 'threw $e', + ), + ]; + await write; + problems.addAll((await Future.wait(reads)).whereType()); + } + expect(problems, isEmpty); + }); + + test( + 'a session file that cannot be read is no session, not an error', + () async { + await store.storeSession('sid', {'user': 1}); + File(store.filePathFor('sid')).writeAsStringSync('not a session'); + store.invalidateCache(); + expect(await store.retrieveSession('sid'), isNull); + }, + ); + + test('no temporary files are left behind', () async { + for (var i = 0; i < 10; i++) { + await store.storeSession('sid', {'n': i}); + } + expect(sandbox.listSync(), hasLength(1)); + }); +} diff --git a/packages/core/test/unit/session_cache_isolates_test.dart b/packages/core/test/unit/session_cache_isolates_test.dart new file mode 100644 index 0000000..02bd371 --- /dev/null +++ b/packages/core/test/unit/session_cache_isolates_test.dart @@ -0,0 +1,44 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; + +void main() { + late Directory sandbox; + + setUp(() { + Env().load(file: File('test/.env')); + // Read once, on first use, so it is set before anything reads it. + Env().env['APP_ISOLATES'] = '2'; + sandbox = Directory.systemTemp.createTempSync('vania_session_isolates'); + SessionFileStore() + ..sessionPath = sandbox.path + ..invalidateCache(); + }); + + tearDown(() { + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + test('with several server isolates, sessions are always read from disk, ' + 'so a change made by another isolate is seen', () async { + final store = SessionFileStore(); + await store.storeSession('sid', {'user': 'a'}); + expect(store.cacheSize, 0); + + // What another isolate's write looks like from here: the file changes + // and this isolate is not told. + final other = File(store.filePathFor('sid')); + final before = other.readAsStringSync(); + await store.storeSession('sid', {'user': 'b'}); + final after = other.readAsStringSync(); + other.writeAsStringSync(before); + expect(await store.retrieveSession('sid'), {'user': 'a'}); + other.writeAsStringSync(after); + expect(await store.retrieveSession('sid'), {'user': 'b'}); + }); +} diff --git a/packages/core/test/unit/session_fixation_test.dart b/packages/core/test/unit/session_fixation_test.dart new file mode 100644 index 0000000..271d284 --- /dev/null +++ b/packages/core/test/unit/session_fixation_test.dart @@ -0,0 +1,134 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_scope.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_manager.dart'; + +/// Serves one request with [handler] and returns the session id the +/// response hands the browser, or the one it came with when it hands none. +/// +/// The session is started first, as for a browser asking for a page, +/// unless [started] is false: then [handler] meets the cookie as an API +/// request does. +Future _visit( + String sessionId, + Future Function(SessionManager sessions) handler, { + bool started = true, +}) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen((req) { + runInRequestScope(req, () async { + final sessions = SessionManager(); + if (started) await sessions.sessionStart(req, req.response); + await handler(sessions); + await Response.json({}).makeResponse(req.response); + }); + }); + final client = HttpClient(); + final rq = await client.get('127.0.0.1', server.port, '/'); + rq.cookies.add(Cookie(SessionManager().sessionKey, sessionId)); + final res = await rq.close(); + await res.drain(); + client.close(force: true); + await server.close(force: true); + return res.cookies + .where((c) => c.name == SessionManager().sessionKey) + .lastOrNull + ?.value ?? + sessionId; +} + +void main() { + late Directory sandbox; + late SessionFileStore store; + + setUpAll(() { + Env().load(file: File('test/.env')); + store = SessionFileStore(); + }); + + setUp(() { + sandbox = Directory.systemTemp.createTempSync('vania_session_fixation'); + store + ..sessionPath = sandbox.path + ..invalidateCache(); + }); + + tearDown(() { + store + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + Future?> stored(String id) { + store.invalidateCache(); + return store.retrieveSession(id); + } + + test('an id the server never issued is not taken up', () async { + final id = await _visit('chosen-by-an-attacker', (_) async {}); + expect(id, isNot('chosen-by-an-attacker')); + expect(await stored('chosen-by-an-attacker'), isNull); + }); + + test('an existing session keeps its id', () async { + await store.storeSession('issued', {'user': 1}); + expect(await _visit('issued', (_) async {}), 'issued'); + }); + + test('nor is it taken up by a request that never started a session, ' + 'such as a fetch to an API route', () async { + final id = await _visit( + 'chosen-by-an-attacker', + (sessions) => sessions.setSession('user', 7), + started: false, + ); + expect(id, isNot('chosen-by-an-attacker')); + expect(await stored('chosen-by-an-attacker'), isNull); + expect(await stored(id), containsPair('user', 7)); + }); + + test('such a request still writes to a session the server issued', () async { + await store.storeSession('issued', {'cart': 3}); + final id = await _visit( + 'issued', + (sessions) => sessions.setSession('user', 7), + started: false, + ); + expect(id, 'issued'); + expect(await stored('issued'), {'cart': 3, 'user': 7}); + }); + + test( + 'regenerating moves the session to a new id and ends the old one', + () async { + await store.storeSession('before-login', {'cart': 3}); + + final id = await _visit('before-login', (sessions) async { + await sessions.regenerateSession(); + await sessions.setSession('user', 7); + }); + + expect(id, isNot('before-login')); + expect(await stored(id), containsPair('cart', 3)); + expect(await stored(id), containsPair('user', 7)); + expect(await stored('before-login'), isNull); + }, + ); + + test('changes made before regenerating go to the new id', () async { + await store.storeSession('before-login', {'cart': 3}); + + final id = await _visit('before-login', (sessions) async { + await sessions.setSession('user', 7); + await sessions.regenerateSession(); + }); + + expect(await stored(id), containsPair('user', 7)); + expect(await stored('before-login'), isNull); + }); +} diff --git a/packages/core/test/unit/session_gate_test.dart b/packages/core/test/unit/session_gate_test.dart index 242b0aa..6510fff 100644 --- a/packages/core/test/unit/session_gate_test.dart +++ b/packages/core/test/unit/session_gate_test.dart @@ -18,6 +18,8 @@ Future _hammerWithHtmlAccept({int count = 5}) async { if (isHtml) { await runInRequestScope(req, () async { await SessionManager().sessionStart(req, req.response); + // RequestHandler saves session writes once the request is done. + await currentRequestScope!.runBeforeResponse(); }); } await req.response.close(); @@ -74,8 +76,9 @@ void main() { }); test('is true once the app has a view directory', () { - Directory('${sandbox.path}/lib/resources/view') - .createSync(recursive: true); + Directory( + '${sandbox.path}/lib/resources/view', + ).createSync(recursive: true); resetSessionGate(); expect(sessionsEnabled(), isTrue); }); @@ -90,8 +93,9 @@ void main() { }); test('SESSION_ENABLED overrides the inference', () { - Directory('${sandbox.path}/lib/resources/view') - .createSync(recursive: true); + Directory( + '${sandbox.path}/lib/resources/view', + ).createSync(recursive: true); Env().env['SESSION_ENABLED'] = 'false'; addTearDown(() => Env().env.remove('SESSION_ENABLED')); resetSessionGate(); @@ -100,11 +104,13 @@ void main() { }); group('API-only app', () { - test('writes no session files however many html-Accept requests arrive', - () async { - await _hammerWithHtmlAccept(); - expect(sessionFileCount(), isZero); - }); + test( + 'writes no session files however many html-Accept requests arrive', + () async { + await _hammerWithHtmlAccept(); + expect(sessionFileCount(), isZero); + }, + ); test('creates no session storage directory at all', () async { await _hammerWithHtmlAccept(); @@ -114,8 +120,9 @@ void main() { group('web app', () { setUp(() { - Directory('${sandbox.path}/lib/resources/view') - .createSync(recursive: true); + Directory( + '${sandbox.path}/lib/resources/view', + ).createSync(recursive: true); resetSessionGate(); }); diff --git a/packages/core/test/unit/session_lifetime_test.dart b/packages/core/test/unit/session_lifetime_test.dart new file mode 100644 index 0000000..7938069 --- /dev/null +++ b/packages/core/test/unit/session_lifetime_test.dart @@ -0,0 +1,131 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_scope.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_manager.dart'; + +/// A returning browser's session: it already has a CSRF token, so a visit +/// writes nothing on that account. +const _returning = {'user': 1, 'x_csrf_token': 'token', 'x_csrf_iv': 'iv'}; + +/// Visits once with [sessionId], the way RequestHandler runs a request, +/// and returns the cookies the response sets. +Future> _visit(String sessionId) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen((req) { + runInRequestScope(req, () async { + await SessionManager().sessionStart(req, req.response); + await Response.json({}).makeResponse(req.response); + }); + }); + final client = HttpClient(); + final rq = await client.get('127.0.0.1', server.port, '/'); + rq.cookies + ..add(Cookie(SessionManager().sessionKey, sessionId)) + ..add(Cookie('XSRF-TOKEN', 'cookie')); + final res = await rq.close(); + await res.drain(); + client.close(force: true); + await server.close(force: true); + return res.cookies; +} + +void main() { + late Directory sandbox; + late SessionFileStore store; + + setUpAll(() { + Env().load(file: File('test/.env')); + // Read once, on first use, so it is set before anything reads it. + Env().env['SESSION_LIFETIME'] = '4'; + store = SessionFileStore(); + }); + + setUp(() { + sandbox = Directory.systemTemp.createTempSync('vania_session_lifetime'); + store + ..sessionPath = sandbox.path + ..invalidateCache(); + }); + + tearDown(() { + store + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + Future expiryOf(String sessionId) async { + store.invalidateCache(); + return (await store.retrieveSessionEntry(sessionId))!.expiresAt; + } + + test('a session lasts SESSION_LIFETIME on the server too', () async { + await store.storeSession('sid', {'user': 1}); + final left = (await expiryOf('sid')).difference(DateTime.now()); + expect(left, greaterThan(const Duration(seconds: 3))); + expect(left, lessThanOrEqualTo(const Duration(seconds: 4))); + }); + + test('a session in use past half its lifetime is extended', () async { + await store.storeSession('sid', _returning); + final before = await expiryOf('sid'); + await Future.delayed(const Duration(milliseconds: 2200)); + + await _visit('sid'); + + final after = await expiryOf('sid'); + expect(after.isAfter(before), isTrue); + expect((await store.retrieveSession('sid'))?['user'], 1); + }); + + test('a fresh session is not written again for being read', () async { + await store.storeSession('sid', _returning); + final file = File(store.filePathFor('sid')); + final written = file.lastModifiedSync(); + await Future.delayed(const Duration(milliseconds: 1100)); + + await _visit('sid'); + + expect(file.lastModifiedSync(), written); + }); + + test('the XSRF cookie lasts as long as the session', () async { + Duration? xsrfLeft(List cookies) => cookies + .where((c) => c.name == 'XSRF-TOKEN') + .lastOrNull + ?.expires + ?.difference(DateTime.now()); + + // A new session, with a token made for it. + final fresh = xsrfLeft(await _visit('unknown')); + expect(fresh, greaterThan(const Duration(seconds: 2))); + expect(fresh, lessThanOrEqualTo(const Duration(seconds: 4))); + + // A returning browser gets it again on every page, as it does the + // session cookie, so the two run out together. + await store.storeSession('sid', _returning); + final renewed = xsrfLeft(await _visit('sid')); + expect(renewed, greaterThan(const Duration(seconds: 2))); + expect(renewed, lessThanOrEqualTo(const Duration(seconds: 4))); + }); + + test('expired session files and stray temporary files are swept', () async { + await store.storeSession('live', {'n': 1}); + await store.storeSession('dead', {'n': 2}); + final dead = File(store.filePathFor('dead')) + ..setLastModifiedSync(DateTime.now().subtract(const Duration(hours: 1))); + final stray = File('${store.filePathFor('live')}.tmp-abc') + ..writeAsStringSync('half') + ..setLastModifiedSync(DateTime.now().subtract(const Duration(hours: 1))); + + await store.sweepExpired(); + + expect(dead.existsSync(), isFalse); + expect(stray.existsSync(), isFalse); + expect(File(store.filePathFor('live')).existsSync(), isTrue); + }); +} diff --git a/packages/core/test/unit/session_write_test.dart b/packages/core/test/unit/session_write_test.dart new file mode 100644 index 0000000..be8701f --- /dev/null +++ b/packages/core/test/unit/session_write_test.dart @@ -0,0 +1,154 @@ +import 'dart:async'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_scope.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/http/session/session_file_store.dart'; +import 'package:vania/src/http/session/session_manager.dart'; + +/// Serves each request through [handlers], keyed by path, inside a request +/// scope with a started session — the way RequestHandler runs a request. +Future _serve( + Map Function(SessionManager s, HttpResponse res)> + handlers, +) async { + final server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen((req) { + runInRequestScope(req, () async { + final sessions = SessionManager(); + await sessions.sessionStart(req, req.response); + await handlers[req.uri.path]!(sessions, req.response); + }); + }); + return server; +} + +/// A browser holding [sessionId]: one the server issued, since it takes +/// up no other. +Future _get(HttpServer server, String path, String sessionId) async { + if (await SessionFileStore().retrieveSession(sessionId) == null) { + await SessionFileStore().storeSession(sessionId, {}); + } + final client = HttpClient(); + final rq = await client.get('127.0.0.1', server.port, path); + rq.cookies.add(Cookie(SessionManager().sessionKey, sessionId)); + await (await rq.close()).drain(); + client.close(force: true); +} + +/// What is on disk for [sessionId], bypassing the in-memory cache. +Future?> _stored(String sessionId) { + SessionFileStore().invalidateCache(); + return SessionFileStore().retrieveSession(sessionId); +} + +void main() { + late Directory sandbox; + late HttpServer server; + + setUp(() { + Env().load(file: File('test/.env')); + sandbox = Directory.systemTemp.createTempSync('vania_session_write'); + SessionFileStore() + ..sessionPath = sandbox.path + ..invalidateCache(); + }); + + tearDown(() async { + await server.close(force: true); + SessionFileStore() + ..sessionPath = 'storage/framework/sessions' + ..invalidateCache(); + sandbox.deleteSync(recursive: true); + }); + + test('writes land on disk together, just before the response', () async { + Map? midRequest; + server = await _serve({ + '/set': (s, res) async { + await s.setSession('a', 1); + await s.setSession('b', 2); + expect(await s.getSession('a'), 1, reason: 'read your writes'); + midRequest = await _stored('sid-1'); + await Response.json({}).makeResponse(res); + }, + }); + + await _get(server, '/set', 'sid-1'); + + expect(midRequest?['a'], isNull, reason: 'nothing written mid-request'); + expect(await _stored('sid-1'), containsPair('a', 1)); + expect(await _stored('sid-1'), containsPair('b', 2)); + }); + + test('a concurrent request\'s writes to the same session survive', () async { + final aHasWritten = Completer(); + final releaseA = Completer(); + server = await _serve({ + '/a': (s, res) async { + await s.setSession('from_a', 1); + aHasWritten.complete(); + await releaseA.future; + await Response.json({}).makeResponse(res); + }, + '/b': (s, res) async { + await s.setSession('from_b', 2); + await Response.json({}).makeResponse(res); + }, + }); + + final a = _get(server, '/a', 'sid-2'); + await aHasWritten.future; + await _get(server, '/b', 'sid-2'); + releaseA.complete(); + await a; + + final stored = await _stored('sid-2'); + expect(stored, containsPair('from_a', 1)); + expect(stored, containsPair('from_b', 2)); + }); + + test('deleting and destroying are saved too', () async { + server = await _serve({ + '/fill': (s, res) async { + await s.setSession('keep', 1); + await s.setSession('drop', 2); + await Response.json({}).makeResponse(res); + }, + '/delete': (s, res) async { + await s.deleteSession('drop'); + await Response.json({}).makeResponse(res); + }, + '/destroy': (s, res) async { + await s.destroyAllSessions(); + await s.setSession('fresh', 3); + expect(await s.getSession('keep'), isNull); + await Response.json({}).makeResponse(res); + }, + }); + + await _get(server, '/fill', 'sid-3'); + await _get(server, '/delete', 'sid-3'); + final afterDelete = await _stored('sid-3'); + expect(afterDelete, containsPair('keep', 1)); + expect(afterDelete!.containsKey('drop'), isFalse); + + await _get(server, '/destroy', 'sid-3'); + expect(await _stored('sid-3'), {'fresh': 3}); + }); + + test('a response written without Response is still saved after', () async { + server = await _serve({ + '/raw': (s, res) async { + await s.setSession('raw', true); + await currentRequestScope!.runBeforeResponse(); + await res.close(); + }, + }); + + await _get(server, '/raw', 'sid-4'); + expect(await _stored('sid-4'), containsPair('raw', true)); + }); +} diff --git a/packages/core/test/unit/static_cache_test.dart b/packages/core/test/unit/static_cache_test.dart index 05e9d76..ed9c986 100644 --- a/packages/core/test/unit/static_cache_test.dart +++ b/packages/core/test/unit/static_cache_test.dart @@ -63,11 +63,13 @@ void main() { expect(res.bodyBytes, greaterThan(0)); expect(res.headers.value(HttpHeaders.etagHeader), isNotNull); expect(res.headers.value(HttpHeaders.lastModifiedHeader), isNotNull); - expect(res.headers.value(HttpHeaders.cacheControlHeader), equals('no-cache')); + expect( + res.headers.value(HttpHeaders.cacheControlHeader), + equals('no-cache'), + ); }); - test('re-requesting with the returned ETag yields 304 and no body', - () async { + test('re-requesting with the returned ETag yields 304 and no body', () async { final first = await _get('app.css'); final etag = first.headers.value(HttpHeaders.etagHeader)!; @@ -123,9 +125,7 @@ void main() { // mtime pair the tag is built from. final file = File('${sandbox.path}/public/app.css'); file.writeAsStringSync('body { color: blue; } /* longer now */'); - file.setLastModifiedSync( - DateTime.now().add(const Duration(seconds: 5)), - ); + file.setLastModifiedSync(DateTime.now().add(const Duration(seconds: 5))); final second = await _get( 'app.css', diff --git a/packages/core/test/unit/static_path_decode_test.dart b/packages/core/test/unit/static_path_decode_test.dart new file mode 100644 index 0000000..198b961 --- /dev/null +++ b/packages/core/test/unit/static_path_decode_test.dart @@ -0,0 +1,97 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/set_static_path.dart'; + +/// Files in `public/` are named in text; their URLs arrive percent-encoded. +/// Decoding must never turn an escape into a way out of `public/`. +void main() { + late Directory sandbox; + late HttpServer server; + + setUpAll(() async { + sandbox = Directory.systemTemp.createTempSync('vania_static_decode'); + final public = Directory('${sandbox.path}/public')..createSync(); + File('${public.path}/my file.css').writeAsStringSync('spaced'); + File('${public.path}/گزارش.txt').writeAsStringSync('persian'); + File('${public.path}/.env').writeAsStringSync('APP_KEY=secret'); + Directory('${public.path}/docs').createSync(); + File('${public.path}/docs/a.txt').writeAsStringSync('nested'); + File('${sandbox.path}/secret.txt').writeAsStringSync('outside public'); + staticPublicRoot = public.path; + + clearRouteCaches(); + initializeRoutes(); + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + }); + + tearDownAll(() async { + await server.close(force: true); + clearRouteCaches(); + staticPublicRoot = 'public'; + sandbox.deleteSync(recursive: true); + }); + + /// Sends [rawPath] exactly as written. + Future<(int, String)> get(String rawPath) async { + final socket = await Socket.connect('127.0.0.1', server.port); + socket.write( + 'GET $rawPath HTTP/1.1\r\nHost: x\r\nAccept: application/json\r\n' + 'Connection: close\r\n\r\n', + ); + final raw = utf8.decode(await socket.expand((c) => c).toList()); + socket.destroy(); + return ( + int.parse(raw.split(' ')[1]), + raw.substring(raw.indexOf('\r\n\r\n') + 4), + ); + } + + test('a file whose name has a space is served', () async { + expect(await get('/my%20file.css'), (200, 'spaced')); + }); + + test('a file with a Persian name is served', () async { + expect(await get('/${Uri.encodeComponent('گزارش.txt')}'), (200, 'persian')); + }); + + test('a nested file is still served', () async { + expect(await get('/docs/a.txt'), (200, 'nested')); + }); + + group('an escape never leads out of public/', () { + for (final path in [ + '/%2E%2E/secret.txt', + '/%2e%2e/secret.txt', + '/docs/%2E%2E/%2E%2E/secret.txt', + '/docs%2F..%2F..%2Fsecret.txt', + '/..%5Csecret.txt', + '/docs%5C..%5C..%5Csecret.txt', + ]) { + test(path, () async { + final (status, body) = await get(path); + expect(body, isNot(contains('outside public'))); + expect(status, isNot(200)); + }); + } + }); + + group('an escape never reaches a hidden file', () { + for (final path in ['/%2Eenv', '/%2eenv', '/docs/%2E%2E/%2Eenv']) { + test(path, () async { + final (status, body) = await get(path); + expect(body, isNot(contains('APP_KEY'))); + expect(status, isNot(200)); + }); + } + }); + + test('a NUL byte or a malformed escape is no file', () async { + expect((await get('/my%20file.css%00.png')).$1, isNot(200)); + expect((await get('/%E0%A4%A.txt')).$1, isNot(200)); + }); +} diff --git a/packages/core/test/unit/throttle_middleware_test.dart b/packages/core/test/unit/throttle_middleware_test.dart new file mode 100644 index 0000000..a9a8438 --- /dev/null +++ b/packages/core/test/unit/throttle_middleware_test.dart @@ -0,0 +1,103 @@ +import 'dart:io'; + +import 'package:test/test.dart'; +import 'package:vania/env.dart'; +import 'package:vania/src/http/request/request_handler.dart'; +import 'package:vania/src/http/response/response.dart'; +import 'package:vania/src/route/middleware/throttle.dart'; +import 'package:vania/src/route/route_handler.dart'; +import 'package:vania/src/route/router.dart'; + +void main() { + late HttpServer server; + late HttpClient client; + + setUpAll(() async { + clearRouteCaches(); + Router.basePrefix(null); + // Each test gets its own route, so its own counter. + for (final path in ['/counted', '/reset', '/default']) { + Router.get( + path, + () => Response.json({}), + ).middleware([Throttle(maxAttempts: 3)]); + } + for (final path in ['/local', '/development']) { + Router.get( + path, + () => Response.json({}), + ).middleware([Throttle(maxAttempts: 3, bypassInDevelopment: true)]); + } + initializeRoutes(); + server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0); + server.listen(RequestHandler().handle); + client = HttpClient(); + }); + + tearDownAll(() async { + client.close(force: true); + await server.close(force: true); + clearRouteCaches(); + }); + + setUp(() => Env().env['APP_ENV'] = 'production'); + tearDown(() => Env().env.remove('APP_ENV')); + + Future get(String path) async { + final rq = await client.get('127.0.0.1', server.port, path); + rq.headers.set('accept', 'application/json'); + final res = await rq.close(); + await res.drain(); + return res; + } + + test('Remaining counts the request it is sent with', () async { + final remaining = []; + final statuses = []; + for (var i = 0; i < 4; i++) { + final res = await get('/counted'); + statuses.add(res.statusCode); + remaining.add(res.headers.value('x-ratelimit-remaining')); + } + expect(statuses, [200, 200, 200, 429]); + expect(remaining, ['2', '1', '0', '0']); + }); + + test( + 'Reset is when the window ends, the same on every request in it', + () async { + final first = (await get('/reset')).headers.value('x-ratelimit-reset'); + await Future.delayed(const Duration(milliseconds: 50)); + final second = (await get('/reset')).headers.value('x-ratelimit-reset'); + expect(second, first); + final resetAt = DateTime.parse(first!); + final untilReset = resetAt.difference(DateTime.now()); + expect(untilReset, lessThanOrEqualTo(const Duration(minutes: 1))); + expect(untilReset, greaterThan(const Duration(seconds: 55))); + }, + ); + + test('APP_ENV=local alone does not turn throttling off', () async { + // What a `.env` copied from a development machine says. + Env().env['APP_ENV'] = 'local'; + final statuses = [ + for (var i = 0; i < 4; i++) (await get('/default')).statusCode, + ]; + expect(statuses, [200, 200, 200, 429]); + }); + + test('bypassInDevelopment skips it for a local app', () async { + // `local` is the documented development value of APP_ENV. + Env().env['APP_ENV'] = 'local'; + for (var i = 0; i < 5; i++) { + expect((await get('/local')).statusCode, 200); + } + }); + + test('and for one that calls its environment development', () async { + Env().env['APP_ENV'] = 'development'; + for (var i = 0; i < 5; i++) { + expect((await get('/development')).statusCode, 200); + } + }); +} diff --git a/packages/core/test/unit/uncaught_errors_test.dart b/packages/core/test/unit/uncaught_errors_test.dart new file mode 100644 index 0000000..ad291d9 --- /dev/null +++ b/packages/core/test/unit/uncaught_errors_test.dart @@ -0,0 +1,41 @@ +import 'dart:async'; +import 'dart:isolate'; + +import 'package:test/test.dart'; +import 'package:vania/src/server/uncaught_errors.dart'; + +/// An isolate as a server runs it: it leaves an error unhandled, then +/// reports whether it is still there to serve. +Future _server((SendPort, bool) args) async { + final (report, guarded) = args; + if (guarded) keepServingOnUncaughtErrors(log: false); + unawaited(Future.error(StateError('nobody awaited me'))); + await Future.delayed(const Duration(milliseconds: 100)); + report.send('still serving'); +} + +Future _run({required bool guarded}) async { + final messages = ReceivePort(); + final exited = ReceivePort(); + await Isolate.spawn(_server, ( + messages.sendPort, + guarded, + ), onExit: exited.sendPort); + final outcome = await Future.any([ + messages.first, + exited.first.then((_) => 'exited'), + ]); + messages.close(); + exited.close(); + return outcome; +} + +void main() { + test('without the guard, an unhandled error stops the isolate', () async { + expect(await _run(guarded: false), 'exited'); + }); + + test('with it, the isolate goes on serving', () async { + expect(await _run(guarded: true), 'still serving'); + }); +} diff --git a/packages/core/test/unit/websocket_dispatcher_test.dart b/packages/core/test/unit/websocket_dispatcher_test.dart index 54ca896..db266fe 100644 --- a/packages/core/test/unit/websocket_dispatcher_test.dart +++ b/packages/core/test/unit/websocket_dispatcher_test.dart @@ -39,45 +39,49 @@ void main() { await server.close(force: true); }); - test('with no driver installed, an upgrade is refused rather than served', - () async { - final status = Completer(); - server.listen((req) async { - await WebSocketUpgradeDispatcher().dispatch(req); - if (!status.isCompleted) status.complete(req.response.statusCode); - }); - - _sendUpgrade(server); - - // Core ships no WebSocket implementation; without a driver such as - // vania_websocket there is nothing to hand the socket to. - expect(await status.future, equals(HttpStatus.notImplemented)); - }); - - test('a handler that throws surfaces the error to the awaiting caller', - () async { - WebSocketUpgradeDispatcher().override((req) async { - throw StateError('handler blew up'); - }); - - final caught = Completer(); - server.listen((req) async { - // Mirrors the WebSocket branch of RequestHandler.handle: the error has - // to be catchable there, otherwise it escapes as an unhandled async - // error with no response left to write. - Object? error; - try { + test( + 'with no driver installed, an upgrade is refused rather than served', + () async { + final status = Completer(); + server.listen((req) async { await WebSocketUpgradeDispatcher().dispatch(req); - } catch (e) { - error = e; - } - if (!caught.isCompleted) caught.complete(error); - }); + if (!status.isCompleted) status.complete(req.response.statusCode); + }); - _sendUpgrade(server); + _sendUpgrade(server); - expect(await caught.future, isA()); - }); + // Core ships no WebSocket implementation; without a driver such as + // vania_websocket there is nothing to hand the socket to. + expect(await status.future, equals(HttpStatus.notImplemented)); + }, + ); + + test( + 'a handler that throws surfaces the error to the awaiting caller', + () async { + WebSocketUpgradeDispatcher().override((req) async { + throw StateError('handler blew up'); + }); + + final caught = Completer(); + server.listen((req) async { + // Mirrors the WebSocket branch of RequestHandler.handle: the error has + // to be catchable there, otherwise it escapes as an unhandled async + // error with no response left to write. + Object? error; + try { + await WebSocketUpgradeDispatcher().dispatch(req); + } catch (e) { + error = e; + } + if (!caught.isCompleted) caught.complete(error); + }); + + _sendUpgrade(server); + + expect(await caught.future, isA()); + }, + ); test('an installed handler receives the upgrade', () async { final seen = Completer(); diff --git a/packages/vania_swagger/CHANGELOG.md b/packages/vania_swagger/CHANGELOG.md index cddcb07..64213b7 100644 --- a/packages/vania_swagger/CHANGELOG.md +++ b/packages/vania_swagger/CHANGELOG.md @@ -1,3 +1,19 @@ +## 1.1.0 + +- The generated spec is now **OpenAPI 3.2.0** (it was 3.1.0). If you feed the + spec to other tools, make sure they understand 3.2; Swagger UI does from + 5.32 on. +- Support for the new HTTP `QUERY` method (RFC 10008). Use + `swaggerQuery(...)`, or `registerRoute(method: 'QUERY')`, and the operation + lands in the path's `query` field. There's also an `@ApiQueryMethod` + annotation next to `@ApiGet`, `@ApiPost` and the rest. `@ApiQuery` hasn't + changed: it still describes a query-string parameter. +- Methods OpenAPI has no field for, like `PURGE`, are written under the + path's `additionalOperations`. Before, they ended up as top-level keys on + the path, which isn't valid in any OpenAPI version. +- Swagger UI bumped from 5.11.0 to 5.33.3. It renders 3.2 specs and can send + `QUERY` requests from "Try it out". + ## 1.0.0 First stable release. diff --git a/packages/vania_swagger/README.md b/packages/vania_swagger/README.md index abeb0eb..d7a3a29 100644 --- a/packages/vania_swagger/README.md +++ b/packages/vania_swagger/README.md @@ -2,7 +2,7 @@ **Describe your API once, and serve interactive OpenAPI 3 docs from your own app.** -`vania_swagger` turns a description of your endpoints into a full OpenAPI 3.0 spec and a live Swagger UI, served on your app's own HTTP port. You write the description with a set of small helper functions — one per endpoint, plus reusable schemas and tags — and the provider does the rest: `/docs` for the UI, and the raw spec as JSON and YAML. +`vania_swagger` turns a description of your endpoints into a full OpenAPI 3.2 spec and a live Swagger UI, served on your app's own HTTP port. You write the description with a set of small helper functions — one per endpoint, plus reusable schemas and tags — and the provider does the rest: `/docs` for the UI, and the raw spec as JSON and YAML. ## Install @@ -60,6 +60,7 @@ GET /docs/swagger.yaml → the spec (YAML) ## Good to know +- HTTP `QUERY` routes are documented with `swaggerQuery`, in the `query` field OpenAPI 3.2 added. (`ApiQuery` stays the query-string parameter; the verb annotation is `ApiQueryMethod`.) - Typed response helpers (`ApiOkResponse`, `ApiCreatedResponse`, `ApiNotFoundResponse`, `ApiUnauthorizedResponse`, …) map to the right status codes. - `swaggerAddBearerAuth()` plus `security:` on a route adds the "Authorize" button so you can try protected endpoints. - The docs *describe* your API — they don't validate it. Keep them honest with a test that generates the spec and asserts on it. diff --git a/packages/vania_swagger/lib/src/annotations/api_operation.dart b/packages/vania_swagger/lib/src/annotations/api_operation.dart index 30d6cca..7d5a4f4 100644 --- a/packages/vania_swagger/lib/src/annotations/api_operation.dart +++ b/packages/vania_swagger/lib/src/annotations/api_operation.dart @@ -398,3 +398,11 @@ class ApiPatch extends ApiRoute { class ApiDelete extends ApiRoute { const ApiDelete(String path) : super('DELETE', path); } + +/// An HTTP `QUERY` route (RFC 10008): a safe, idempotent read whose query +/// is in the request content. +/// +/// Not `ApiQuery`, which is taken by query-string parameters. +class ApiQueryMethod extends ApiRoute { + const ApiQueryMethod(String path) : super('QUERY', path); +} diff --git a/packages/vania_swagger/lib/src/generator/swagger_generator.dart b/packages/vania_swagger/lib/src/generator/swagger_generator.dart index 55b8197..b058c7f 100644 --- a/packages/vania_swagger/lib/src/generator/swagger_generator.dart +++ b/packages/vania_swagger/lib/src/generator/swagger_generator.dart @@ -1,6 +1,20 @@ import '../metadata/swagger_metadata.dart'; import '../annotations/annotations.dart'; +/// The methods an OpenAPI 3.2 Path Item has a field of its own for. Any +/// other goes under `additionalOperations`. +const Set _pathItemMethods = { + 'get', + 'put', + 'post', + 'delete', + 'options', + 'head', + 'patch', + 'trace', + 'query', +}; + class SwaggerGenerator { static final SwaggerGenerator _instance = SwaggerGenerator._internal(); factory SwaggerGenerator() => _instance; @@ -11,7 +25,7 @@ class SwaggerGenerator { final info = metadata.apiInfo; final spec = { - 'openapi': '3.1.0', + 'openapi': '3.2.0', 'info': _buildInfo(info), if (serverUrl != null || metadata.servers.isNotEmpty) 'servers': _buildServers(metadata, serverUrl), @@ -52,7 +66,15 @@ class SwaggerGenerator { } final operation = _buildOperation(route, securitySchemes); - (paths[path] as Map)[method] = operation; + final pathItem = paths[path] as Map; + if (_pathItemMethods.contains(method)) { + pathItem[method] = operation; + } else { + // OpenAPI 3.2 keys these by the method exactly as it is sent. + (pathItem['additionalOperations'] ??= + {})[route.method] = + operation; + } final tags = route.tags.isNotEmpty ? route.tags diff --git a/packages/vania_swagger/lib/src/generator/yaml_emitter.dart b/packages/vania_swagger/lib/src/generator/yaml_emitter.dart index 403da26..a2b97f6 100644 --- a/packages/vania_swagger/lib/src/generator/yaml_emitter.dart +++ b/packages/vania_swagger/lib/src/generator/yaml_emitter.dart @@ -1,4 +1,4 @@ -/// A tiny YAML emitter that covers what OpenAPI 3.1 needs — maps, lists, +/// A tiny YAML emitter that covers what OpenAPI 3.2 needs — maps, lists, /// scalars (String / num / bool / null). No anchors, no flow style, no /// custom tags. Keys are emitted as unquoted identifiers when safe, /// otherwise as double-quoted strings. diff --git a/packages/vania_swagger/lib/src/http/swagger_controller.dart b/packages/vania_swagger/lib/src/http/swagger_controller.dart index 3810f83..6544bc7 100644 --- a/packages/vania_swagger/lib/src/http/swagger_controller.dart +++ b/packages/vania_swagger/lib/src/http/swagger_controller.dart @@ -25,13 +25,13 @@ class SwaggerController { return Response.html(html, headers: _headers('text/html; charset=utf-8')); } - /// Serves the OpenAPI 3.1 spec as JSON. + /// Serves the OpenAPI 3.2 spec as JSON. Future serveJson(Request request) async { final spec = _spec(request); return Response.jsonWithHeader(spec, headers: _headers('application/json')); } - /// Serves the OpenAPI 3.1 spec as YAML. + /// Serves the OpenAPI 3.2 spec as YAML. Future serveYaml(Request request) async { final spec = _spec(request); final yaml = encodeYaml(spec); diff --git a/packages/vania_swagger/lib/src/http/swagger_page_renderer.dart b/packages/vania_swagger/lib/src/http/swagger_page_renderer.dart index 01e3ab5..4d7e6fe 100644 --- a/packages/vania_swagger/lib/src/http/swagger_page_renderer.dart +++ b/packages/vania_swagger/lib/src/http/swagger_page_renderer.dart @@ -23,7 +23,7 @@ class SwaggerPageRenderer { ${_htmlEscape(title)} - +