Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls - #35
Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls#35edward-rosado wants to merge 3 commits into
Conversation
… calls The 429 handler slept-and-retried inside the failing request with no retry cap and nothing gating other requests, so the poll loop kept firing every 15s, each call collected another 429, and the ban window never cleared — observed live: 134 consecutive 429s over 2h11m while Retry-After counted down 8008s in lockstep, proving the wait was never honored. Replaced with a single global backoff window (rateLimitedUntil): - Parses Retry-After defensively: integer or HTTP-date form; NaN/zero/ negative fall back to 5s. A careless parse yields NaN, and now<NaN is always false, which would silently disable the whole mechanism. - Clamped to 300s (Spotify has sent 13-hour values; one probe per 5min re-arms the window if the ban is truly longer, which beats a silent half-day freeze), plus jitter against a thundering herd of pollers. - Math.max so a concurrent 429 with a shorter Retry-After never shrinks an open window; four pollers run concurrently. - The gate sits after the in-flight queue lookup (join, don't refuse a fresh request already underway), serves cached GETs up to 60s old, resolves undefined to match the contract all ~25 call sites are written against, and logs once per window instead of per poll. Also cut the two biggest sources of the traffic that trips the limit: - checkLiked fired on every 15s poll (any progress advance registered as a state change) — a third of all API calls, and the endpoint that absorbed every recorded 429 and 403. Now memoized per track with a 5-minute TTL, invalidated by likeSong so the toggle stays instant. - The queue cache TTL was 10s under a 15s poll, guaranteeing a live call every cycle; raised to 45s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Found while bringing up the Bluetooth transport for the Car Thing (ItsRiprod/DeskThing#152) — a hard-rate-limited Spotify app looks identical to a broken transport from the device's side, since both show a blank now-playing screen. Client-side companion: ItsRiprod/DeskThing-Client#31 |
The GET queue exists to join callers onto a request that is still in flight. Nothing removed the entry once the request settled, so for a further three seconds every caller was handed the resolved promise and its now-stale payload — a second, hidden cache that no forceRefresh could bypass, because the queue check never looked at that flag. At a track boundary this is expensive. The refresh fired when a track ends asks a beat before Spotify reports the advance, so it sees the track that just finished; its retries are answered from that same settled promise, and the next scheduled poll lands inside the window and is answered from it too. An entire poll cycle reports "no significant state changes" without ever reaching Spotify, which is how a one-second gap turns into two full cycles — the 17.8s worst case measured across 25 transitions. The entry is now dropped when the promise settles, so "in queue" means "in flight" again, and a forced refresh no longer joins one. requestCache still coalesces ordinary rapid GETs, so nothing loses de-duplication. forceRefresh is plumbed from the server's REFRESH request through to getCurrentPlayback, set only when the caller already knows playback changed. The routine cadence keeps its coalescing. Also cache failed liked-lookups briefly. checkLiked returned before writing the memo whenever the call failed, so under a persistent 403 (a token without user-library-read) the memo never populated and every emit re-issued a call that could not succeed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Pushed one more commit here, found while chasing a separate latency bug on real hardware. The GET request queue was answering with a request that had already finished.
That is expensive at a track boundary. The refresh fired when a track ends asks a beat before Spotify reports the advance, so it sees the track that just finished; its retries are answered from that same settled promise, and the next scheduled poll lands inside the window and is answered from it too. An entire poll cycle logs "no significant state changes" without ever reaching Spotify. Reconstructed from the logs at one boundary: track ends 6:43:34, Spotify advances ~6:43:35, poll grid :07/:22/:37/:52. The boundary refresh at :35 got the old track, retried at :36 and :37, and the scheduled :37 poll was served the identical stale payload and vanished. Next real observation: :52. That is 17.8s for a one-second gap, and the tail is structurally unbounded — two swallowed polls in a row would give ~30s. The entry is now dropped when the promise settles, so "in queue" means "in flight" again, and a forced refresh no longer joins one.
Also, a follow-up to the liked-memo in this PR. Server-side effect of this plus the boundary work in ItsRiprod/DeskThing#153: a skip went from ~13-15s to 0.74-0.83s across 8 runs, and a natural track end from a median of ~11s to 0.51s. Caveat worth stating plainly: this repo has no test harness, so unlike the DeskThing-side change these are verified on hardware rather than by tests. I did not add a framework in a fix PR, but I'm happy to if you'd like one. |
App.tsx imported ./providers/PlaylistProvider, which was deleted when playlists moved to a zustand store. Nothing aliases the path, so the app has not compiled since — dist/ is empty, and whatever runs on a device is a prebuilt bundle that has drifted from this source. Every consumer already reaches for usePlaylistStore directly, so the wrapper had nothing left to provide. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
a5cd10b to
92c89c9
Compare
Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls
The bug
The 429 handler slept inside the request that got rate-limited and then retried it in place — but nothing gated other requests. DeskThing's music poller kept firing every 15s, each call collected another 429, and the ban window never cleared. There was also no
retryCount < MAX_RETRIESguard on that path, unlike the 401 path.Captured live on my setup:
Retry-Aftercounted down 8008s → 158s in lockstep with the polling — decrementing by exactly 15 each cycle, which is direct proof the wait was never actually honoredThe fix
A single global backoff window (
rateLimitedUntil), because Spotify's 429 applies to the whole app, not one endpoint:Retry-Afterparsing. RFC 7231 allows an HTTP-date, andparseInt("Wed, 05 Aug 2026 15:00:00 GMT")isNaN— andnow < NaNis always false, which would silently disable the entire mechanism forever. Both forms are parsed; zero/negative/garbage fall back to 5s.Retry-After: 47967(13.3 hours). Unclamped, the app goes silently dead for half a day. One probe request every 5 minutes re-arms the window if the ban really is longer — self-correcting, and far better than a half-day freeze.Math.max, never a plain assignment. Four pollers run concurrently (song, queue, device, playlist). Without this, a 429 carryingRetry-After: 1arriving 50ms after one carrying60collapses the window to 1s and the app instantly re-trips.undefined— the contract all ~25 call sites are already written against — instead of introducing a second failure signal. It also serves cached GETs up to 60s old, so the UI shows the last-known track instead of going blank during a window.Also: the traffic that trips the limit
Two changes cut steady-state API calls by ~3×:
checkLikedwas firing on every single 15s poll.checkForRefreshtreats a progress delta > 3000ms as a state change — but a 15s poll always advances progress by ~15000ms, sostateChangedwas permanently true. That made/me/tracks/containsa third of all API traffic, and it's the endpoint that absorbed every recorded 429 and 403. Now memoized per track (5-minute TTL), invalidated bylikeSongso the like toggle stays instant.Verification
Measured on a live install with music playing, over a 5-minute window after the fix:
/me/tracks/containsThe
Retry-Afterparse is unit-tested out-of-process against"120","0","-1",null,"","47967","garbage", and both future and past HTTP-date forms — every case yields a finite window in(now, now+302s]. The never-shrink property is tested by firing a 60s window followed by a 1s one.Typecheck passes; the server bundles cleanly with esbuild.
🤖 Generated with Claude Code