Repository navigation
Enhance core features with QUERY method, session handling, and logging - #218
Merged
Merged
Conversation
A line like `HTTP_BACKLOG=` in .env broke typed reads such as env<int>(). An empty or blank value now falls back to the default you pass. The logger used to open the log file for every entry. It now keeps one sink per file, never throws (a failed write goes to stderr), and gains Logger.flush() to write out pending lines before the process exits.
- Response.fromPath streams a file from disk instead of loading it into memory, and answers Range requests with 206, so video seeking and resumed downloads work. - Download names outside ASCII are sent with filename* (RFC 8187); a file called گزارش.pdf no longer causes a 500. - Server-sent events are flushed as they happen, kept alive with a heartbeat (SSE_HEARTBEAT) and cancelled when the client goes away. - A body that can't be encoded as JSON is a 500 instead of a broken 200, and Response() with no data sends an empty body. - back() only follows a same-site Referer and doesn't loop on GET. - HTML error responses carry the real status code. - The request scope gets a before-response hook, so pending work such as session writes happens once, right before the response is written.
- Cookies and the body are read on first use. This also fixes CSRF rejecting every POST, because the cookie was read before it was parsed. - MAX_BODY_SIZE is enforced for chunked multipart uploads too. - New Request.rawBody for content that isn't JSON or a form. - has(), except(), headers, file() and files fixed for edge cases. - Uploaded bytes are no longer copied twice on their way to storage.
QUERY (RFC 10008) is a safe, idempotent request that carries its query in
the body:
Router.query('/contacts', search).acceptQuery(['application/json']);
A body of a type the route doesn't take gets a 415 with an Accept-Query
header, and OPTIONS answers with an Allow header for the path.
Routing changes along the way:
- Paths are decoded before matching, so parameters reach the controller
decoded and routes written in Persian or any other script match.
- where() patterns match the whole segment; whereString, whereDouble and
whereBool are enforced and hand the controller the declared type.
- HEAD is answered by the GET route.
- Router.any(...).middleware(...) applies to every method it registered,
and a group that throws no longer leaks its prefix and middleware.
- Router.apiResource registers a resource without the form pages.
- Dynamic routes are looked up through an index, so the route cache and
ROUTE_LOOK_UP_SIZE are gone.
- Static files with spaces or non-Latin names are found, and encoded
path separators can't be used to step outside public/.
Sessions - An id the server never issued is ignored instead of adopted, and the new regenerateSession() moves a session to a fresh id (call it on login). - Sessions live for SESSION_LIFETIME instead of a fixed hour and are extended once past half of it. - Changes are buffered and saved once per request, only the keys that changed. Files are written atomically and a damaged one counts as no session. Expired files are swept hourly. - With several isolates the in-memory cache is turned off, since each isolate would otherwise hold its own copy. Flash messages, validation errors and old input now survive the redirect. CSRF covers DELETE, and csrf_except entries match a path and what's under it (webhooks no longer covers webhooks-admin). Throttle reports X-RateLimit-Remaining after counting the request, sends the real reset time, and is skipped while APP_ENV is local.
The controller returns its Response and the framework writes it once every middleware is done, so a middleware can still add headers after await next(), and an error from the controller comes back out of next(). - One error policy for errors thrown anywhere in the chain. Apps without views answer in JSON; a 5xx message is only shown with APP_DEBUG; Unauthenticated redirects only when asked to with ResponseType.html. - CSRF runs first, before route middleware. - Calling next() twice throws instead of running the controller again, and a middleware that neither calls next() nor answers gets a 500 instead of leaving the request open. - REQUEST_TIMEOUT answers with a 503; whatever the handler does after that never reaches the client or the session. - Error pages are rendered with their status. - handle() never throws, even when the response has already started.
- APP_ISOLATES (a number or `auto`) runs the server in that many isolates on a shared socket. An isolate that dies is replaced, with a backoff. - SIGINT and SIGTERM stop accepting connections and give requests in flight up to SHUTDOWN_TIMEOUT seconds to finish. - New HTTP_BACKLOG and HTTP_IDLE_TIMEOUT settings. - An error nobody handles, like a failing future nobody awaited, is logged instead of stopping the isolate.
Bump the version and write up what changed in the HTTP layer. Also keep the storage/ folder the tests create out of git.
- The generated spec is OpenAPI 3.2.0. - QUERY operations go in the path's `query` field, via swaggerQuery(...) or registerRoute(method: 'QUERY'), and there's a new @ApiQueryMethod annotation. @apiquery still means a query-string parameter. - Methods OpenAPI has no field for, like PURGE, are written under additionalOperations instead of as invalid top-level path keys. - Swagger UI goes from 5.11.0 to 5.33.3, which renders 3.2 and can send QUERY from "Try it out".
Configuration lists the new settings (APP_ISOLATES, REQUEST_TIMEOUT, SHUTDOWN_TIMEOUT, HTTP_BACKLOG, HTTP_IDLE_TIMEOUT, SSE_HEARTBEAT). Routing, requests, responses, middleware, sessions, validation and error handling are updated for QUERY, HEAD, rawBody, fromPath and Range, the new middleware order, regenerateSession() and the one error policy. The swagger page describes OpenAPI 3.2 and QUERY.
A small app with one route per feature: QUERY, HEAD, Range downloads, SSE, CSRF, sessions and flash messages, error pages, throttling and multi-isolate serving. Open /console in a browser for checks that run in the page, or run `dart run tool/check.dart` against a running server. `--isolates` starts its own server to test isolates and shutdown.
sessionStart already refuses a session id the server never issued, but only page requests go through it. A fetch to an API route took the cookie as it was, and the first setSession wrote a new session under that id. Someone who planted the cookie could then ride along into a session created by a JSON login. A request that writes to a session its cookie doesn't point at now gets a fresh id, sent back in the session cookie. Writing to a session the server did issue works as before.
With APP_ISOLATES above 1 the extra isolates boot from the config but
never run main(). Anything an app did there before initialize, like
registering its database driver or setting up auth, only happened in the
first isolate, so the others failed to boot or served requests without
it.
initialize() now takes a `setup` callback that runs in each isolate,
before the providers boot:
await Application().initialize(
config: config,
setup: () => registerMySqlDriver(),
);
Throttle skipped itself whenever APP_ENV was local, and local is what most .env files say, including ones that end up on a server. A production app running on a copied .env had no rate limits and nothing told it so. bypassInDevelopment now defaults to false. Pass true to skip throttling while APP_ENV is local or development.
An isolate that kept dying at startup was retried after 1, 2, 4 ... 32 seconds and then the wait went back to one second, so a broken boot was retried forever on a short loop. The wait now doubles up to a minute and stays there, as the docs already said.
The static route index held one route per path, so registering `/` for admin.example.com and `/` for any host made the later one replace the earlier: one of the two sites lost its page. Each path now keeps one route per domain, routes for a domain are tried before the one for any host, and registering the same path twice for the same host still keeps the later route. Also, a path segment with an encoded slash (`a%2Fb`) was joined back into two segments for the static lookup and could hit the static route `a/b`. It is now looked up as it came, which matches how dynamic routes already treat it.
The XSRF-TOKEN cookie was set once with a fixed 9000 second lifetime, while the session cookie follows SESSION_LIFETIME and is renewed on every page. With a longer lifetime, or a form left open for a few hours, the XSRF cookie ran out first and the form got a 419 although the session was still valid. It is now sent with every page with the session's lifetime.
A large upload over a slow connection can run out of REQUEST_TIMEOUT and get a 503. That's on purpose, since it stops a client from trickling a body in forever, but it wasn't written anywhere.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is mostly about the HTTP layer: everything between a request arriving and the response leaving. It ships as vania 2.1.0 and vania_swagger 1.1.0, both already on pub.dev.
Vania is API-first, and this release leans into that: JSON with a real status code is the default everywhere. Redirects, HTML pages and sessions only happen when the app has views or asks for them.
What's new
APP_ISOLATES(a number orauto) runs the server in several isolates on one port. An isolate that crashes is replaced, with a backoff capped at a minute. The newsetup:callback onApplication().initializeruns in every isolate, for things like registering a database driver.SHUTDOWN_TIMEOUTseconds to finish.QUERYmethod (RFC 10008):Router.query,.acceptQuery([...])with 415 +Accept-Query, and anAllowheader onOPTIONS.HEADis answered by the GET route.Response.fromPathstreams files from disk and supportsRange(206), so seeking and resumed downloads work.Request.rawBody,Router.apiResource,regenerateSession(),Logger.flush().REQUEST_TIMEOUT,HTTP_IDLE_TIMEOUT,HTTP_BACKLOG,SSE_HEARTBEAT,SESSION_LIFETIME.Behaviour changes
await next(). Controller errors come back out ofnext(). Callingnext()twice throws.APP_DEBUG.Unauthenticatedonly redirects when created withResponseType.html.where()patterns match the whole segment, andwhereInt/whereString/whereDouble/whereBoolare enforced.DELETE.csrf_exceptentries are segment-bounded.Throttle's development bypass is now opt-in (bypassInDevelopment: true).ROUTE_LOOK_UP_SIZEis gone. Route lookup is indexed.Fixes
MAX_BODY_SIZEapplies to chunked multipart uploads.Swagger (vania_swagger 1.1.0)
QUERYoperations viaswaggerQuery(...)/@ApiQueryMethod.additionalOperations.Also in this PR
examples/core_showcase: a sample app that exercises every feature above./consoleruns browser checks, anddart run tool/check.dartruns request checks against a live server.Testing
packages/core: 557 tests pass,dart analyzeis clean.packages/vania_swagger: 50 tests pass.examples/core_showcase: 46/46 request checks against a running server, plus 4/4 multi-isolate and shutdown checks.dart pub publish --dry-run: 0 warnings for both packages.