Repository navigation
fix(auth): send verification codes from the blocking pool, not the async worker - #214
Merged
Merged
Conversation
…ync worker `crate::mail::send_verification_code` is blocking I/O: 3 attempts with a 2 s sleep between them, and a 15 s timeout per attempt when SMTP accepts a connection but never answers. It was called in place from three unauthenticated handlers (`register`, `resend-code`, `forgot-password`), so it occupied a tokio **worker** thread instead of the blocking pool — the whole runtime stops serving while it runs. Measured (probe injected, then reverted byte-identically): on a 1-worker runtime one `register` pushed a concurrent request's 100 ms sleep to 4.2427 s; 14 concurrent registers on the default 12 workers pushed it to 4.3416 s; a silently dropping SMTP held one worker for 49.028 s. The whole tree had zero `spawn_blocking` calls. `send_code` is now `async` and moves the send into `tokio::task::spawn_blocking` with owned inputs; a `JoinError` becomes a 500 instead of being swallowed. The SMTP-failure arm (log + `clear_verification` + 502) stays outside the blocking task, so it still takes the DB lock on the async path. `src/mail.rs` is untouched. The DB guard scopes in the three handlers had to become explicit blocks rather than `drop(conn)`: with an `await` after them the generator's `Send` analysis still counts the `MutexGuard` as live across the await. No behaviour change — the guards are released at the same points. Test: `slow_smtp_send_does_not_stall_the_runtime` runs the router on a 1-worker runtime with a real listener and, from outside the runtime, issues `GET /healthz` while a `register` sits inside the blocking send (gated on the fake SMTP's first accept, so the argon2 cost is not part of the window). Fixed: 0.5 ms; with only the `spawn_blocking` reverted: 4.016 s and that test is the only failure (215 passed / 1 failed). Companions: `accepts == 3` on both trees, a dev-mode non-blocking control arm, and `register` still answering 502.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crate::mail::send_verification_codeis blocking I/O (lettre: 3 attempts, a 2 s sleep between them, 15 s timeout per attempt when SMTP accepts a connection and never answers). It was called in place from three unauthenticated handlers, so it occupied a tokio worker thread instead of the blocking pool — while it runs, the whole runtime stops serving (including/healthz, the liveness probe).Measured before the fix, with a probe injected into
src/mail.rs+src/routes/mod.rsand then reverted byte-identically:available_parallelism)registerin flightsleep(100ms)took 4.2427 s — the process stops servingregisterssleep(1500ms)took 4.3416 s,accepts=42(14x3) ⇒ pool saturatedgrep spawn_blockingacross the tree was 0. The author's convention is already stated in the same file —send_code's doc says 「发信(含重试)不能持锁」 (mailing must not block a shared resource) — only the resource it named was the DB mutex; nobody re-checked the worker pool.mail.rsnever mentions the async runtime. Impact is availability, not money: 12 concurrent unauthenticated requests (2 on a 2-core box) stall the gateway, and a failed liveness probe restarts the container.Changes
send_code(src/routes/mod.rs) is nowasyncand moves the send intotokio::task::spawn_blockingwith owned inputs (cfgclone,email/codeasString). AJoinErrorbecomes a 500 rather than being swallowed.clear_verification+ 502) stays outside the blocking task, so it still takes the DB lock on the async path.src/mail.rsis untouched.register,resend-code,forgot-password) now.await.drop(conn). With anawaitafter them, the async-generatorSendanalysis still counts theMutexGuardas live across the await (future cannot be sent between threads safely), so axum'sHandlerbound fails to compile. Guards are released at exactly the same points as before.config/config.example.tomlneeds no update.Related Issue
None — the problem was found by a runtime probe, not an issue report.
Tests
cargo test— 216 passed (was 215; +1 new test), 0 failedcargo fmt --check— clean;cargo clippy --all-targets -- -D warnings— cleanNew test
routes::tests::slow_smtp_send_does_not_stall_the_runtime: runs the router on a 1-worker runtime with a real listener, and from outside the runtime (a std thread, so it cannot be blocked by the runtime itself) issuesGET /healthzwhile aregistersits inside the blocking send against a fake SMTP that accepts-then-closes.healthz_wait = 521 µs. With only thespawn_blockingreverted:healthz_wait = 4.015941458 s, and that test is the only failure (215 passed / 1 failed) — i.e. the assertion discriminates exactly the defect.accepts == 3on both trees (the blocking retry path really ran),registerstill answering 502 with the same body, and a dev-mode control arm (no SMTP ⇒ non-blocking) answering201with a ms-scale/healthz.Checklist
fix/)fix(auth): …)mail.rs, the 502-vs-201 semantics, the DB lock, the schema,ui/, i18n