Skip to content

Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls - #35

Open
edward-rosado wants to merge 3 commits into
ItsRiprod:masterfrom
edward-rosado:fix/spotify-rate-limit-backoff
Open

Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls#35
edward-rosado wants to merge 3 commits into
ItsRiprod:masterfrom
edward-rosado:fix/spotify-rate-limit-backoff

Conversation

@edward-rosado

Copy link
Copy Markdown

Fix Spotify rate limiting: honor Retry-After globally, stop redundant calls

The bug

The 429 handler slept inside the request that got rate-limited and then retried it in place — but nothing gated other requests. DeskThing's music poller kept firing every 15s, each call collected another 429, and the ban window never cleared. There was also no retryCount < MAX_RETRIES guard on that path, unlike the 401 path.

Captured live on my setup:

  • 134 consecutive 429s over 2h11m, at a steady 15.007s interval
  • Spotify's Retry-After counted down 8008s → 158s in lockstep with the polling — decrementing by exactly 15 each cycle, which is direct proof the wait was never actually honored
  • The app served empty song data to the device the whole time (it renders as a blank "undefined cover" screen on a Car Thing)

The fix

A single global backoff window (rateLimitedUntil), because Spotify's 429 applies to the whole app, not one endpoint:

  • NaN-proof Retry-After parsing. RFC 7231 allows an HTTP-date, and parseInt("Wed, 05 Aug 2026 15:00:00 GMT") is NaN — and now < NaN is always false, which would silently disable the entire mechanism forever. Both forms are parsed; zero/negative/garbage fall back to 5s.
  • Clamped to 300s. The logs contain Retry-After: 47967 (13.3 hours). Unclamped, the app goes silently dead for half a day. One probe request every 5 minutes re-arms the window if the ban really is longer — self-correcting, and far better than a half-day freeze.
  • Math.max, never a plain assignment. Four pollers run concurrently (song, queue, device, playlist). Without this, a 429 carrying Retry-After: 1 arriving 50ms after one carrying 60 collapses the window to 1s and the app instantly re-trips.
  • Jitter, so every poller doesn't unblock on the same millisecond when the window lapses.
  • Gate placed after the in-flight queue lookup, so a request already underway is joined rather than refused.
  • Resolves undefined — the contract all ~25 call sites are already written against — instead of introducing a second failure signal. It also serves cached GETs up to 60s old, so the UI shows the last-known track instead of going blank during a window.
  • Logs once per window, not once per gated poll, with a matching "window cleared" line on recovery.

Also: the traffic that trips the limit

Two changes cut steady-state API calls by ~3×:

  • checkLiked was firing on every single 15s poll. checkForRefresh treats a progress delta > 3000ms as a state change — but a 15s poll always advances progress by ~15000ms, so stateChanged was permanently true. That made /me/tracks/contains a third of all API traffic, and it's the endpoint that absorbed every recorded 429 and 403. Now memoized per track (5-minute TTL), invalidated by likeSong so the like toggle stays instant.
  • The queue cache TTL was 10s under a 15s poll, guaranteeing a live call every cycle. Raised to 45s.

Verification

Measured on a live install with music playing, over a 5-minute window after the fix:

before after
API requests 12/min 4/min
/me/tracks/contains every 15s cycle 0 (only on track change)
429s continuous 0

The Retry-After parse is unit-tested out-of-process against "120", "0", "-1", null, "", "47967", "garbage", and both future and past HTTP-date forms — every case yields a finite window in (now, now+302s]. The never-shrink property is tested by firing a 60s window followed by a 1s one.

Typecheck passes; the server bundles cleanly with esbuild.

🤖 Generated with Claude Code

… calls

The 429 handler slept-and-retried inside the failing request with no
retry cap and nothing gating other requests, so the poll loop kept
firing every 15s, each call collected another 429, and the ban window
never cleared — observed live: 134 consecutive 429s over 2h11m while
Retry-After counted down 8008s in lockstep, proving the wait was never
honored.

Replaced with a single global backoff window (rateLimitedUntil):
- Parses Retry-After defensively: integer or HTTP-date form; NaN/zero/
  negative fall back to 5s. A careless parse yields NaN, and now<NaN is
  always false, which would silently disable the whole mechanism.
- Clamped to 300s (Spotify has sent 13-hour values; one probe per 5min
  re-arms the window if the ban is truly longer, which beats a silent
  half-day freeze), plus jitter against a thundering herd of pollers.
- Math.max so a concurrent 429 with a shorter Retry-After never shrinks
  an open window; four pollers run concurrently.
- The gate sits after the in-flight queue lookup (join, don't refuse a
  fresh request already underway), serves cached GETs up to 60s old,
  resolves undefined to match the contract all ~25 call sites are
  written against, and logs once per window instead of per poll.

Also cut the two biggest sources of the traffic that trips the limit:
- checkLiked fired on every 15s poll (any progress advance registered
  as a state change) — a third of all API calls, and the endpoint that
  absorbed every recorded 429 and 403. Now memoized per track with a
  5-minute TTL, invalidated by likeSong so the toggle stays instant.
- The queue cache TTL was 10s under a 15s poll, guaranteeing a live
  call every cycle; raised to 45s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edward-rosado

Copy link
Copy Markdown
Author

Found while bringing up the Bluetooth transport for the Car Thing (ItsRiprod/DeskThing#152) — a hard-rate-limited Spotify app looks identical to a broken transport from the device's side, since both show a blank now-playing screen.

Client-side companion: ItsRiprod/DeskThing-Client#31

The GET queue exists to join callers onto a request that is still in
flight. Nothing removed the entry once the request settled, so for a
further three seconds every caller was handed the resolved promise and
its now-stale payload — a second, hidden cache that no forceRefresh
could bypass, because the queue check never looked at that flag.

At a track boundary this is expensive. The refresh fired when a track
ends asks a beat before Spotify reports the advance, so it sees the
track that just finished; its retries are answered from that same
settled promise, and the next scheduled poll lands inside the window and
is answered from it too. An entire poll cycle reports "no significant
state changes" without ever reaching Spotify, which is how a one-second
gap turns into two full cycles — the 17.8s worst case measured across 25
transitions.

The entry is now dropped when the promise settles, so "in queue" means
"in flight" again, and a forced refresh no longer joins one. requestCache
still coalesces ordinary rapid GETs, so nothing loses de-duplication.

forceRefresh is plumbed from the server's REFRESH request through to
getCurrentPlayback, set only when the caller already knows playback
changed. The routine cadence keeps its coalescing.

Also cache failed liked-lookups briefly. checkLiked returned before
writing the memo whenever the call failed, so under a persistent 403
(a token without user-library-read) the memo never populated and every
emit re-issued a call that could not succeed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@edward-rosado

Copy link
Copy Markdown
Author

Pushed one more commit here, found while chasing a separate latency bug on real hardware.

The GET request queue was answering with a request that had already finished.

requestQueue exists to join a caller onto a request that is still in flight. Nothing removed the entry once the request settled, so for a further cacheTime (3s) every caller was handed the resolved promise and its now-stale payload. It was a second, hidden cache — and one that forceRefresh could not bypass, because the queue check never looked at that flag.

That is expensive at a track boundary. The refresh fired when a track ends asks a beat before Spotify reports the advance, so it sees the track that just finished; its retries are answered from that same settled promise, and the next scheduled poll lands inside the window and is answered from it too. An entire poll cycle logs "no significant state changes" without ever reaching Spotify.

Reconstructed from the logs at one boundary: track ends 6:43:34, Spotify advances ~6:43:35, poll grid :07/:22/:37/:52. The boundary refresh at :35 got the old track, retried at :36 and :37, and the scheduled :37 poll was served the identical stale payload and vanished. Next real observation: :52. That is 17.8s for a one-second gap, and the tail is structurally unbounded — two swallowed polls in a row would give ~30s.

The entry is now dropped when the promise settles, so "in queue" means "in flight" again, and a forced refresh no longer joins one. requestCache still coalesces ordinary rapid GETs, so nothing loses de-duplication.

forceRefresh is plumbed from the server's REFRESH request through to getCurrentPlayback, set only when the caller already knows playback changed. The routine cadence keeps its coalescing.

Also, a follow-up to the liked-memo in this PR. checkLiked returned before writing the memo whenever the call failed. Under the persistent 403 this account gets from /me/tracks/contains (a token without user-library-read) the memo therefore never populated, and every emit re-issued a call that could not succeed — 357 calls across 28 distinct track ids in an 8-hour window, zero cache hits. Failures are now remembered briefly too, so a doomed lookup happens at most once a minute per track rather than on every update.

Server-side effect of this plus the boundary work in ItsRiprod/DeskThing#153: a skip went from ~13-15s to 0.74-0.83s across 8 runs, and a natural track end from a median of ~11s to 0.51s.

Caveat worth stating plainly: this repo has no test harness, so unlike the DeskThing-side change these are verified on hardware rather than by tests. I did not add a framework in a fix PR, but I'm happy to if you'd like one.

App.tsx imported ./providers/PlaylistProvider, which was deleted when
playlists moved to a zustand store. Nothing aliases the path, so the app
has not compiled since — dist/ is empty, and whatever runs on a device is
a prebuilt bundle that has drifted from this source.

Every consumer already reaches for usePlaylistStore directly, so the
wrapper had nothing left to provide.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@edward-rosado
edward-rosado force-pushed the fix/spotify-rate-limit-backoff branch from a5cd10b to 92c89c9 Compare August 7, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant