Repository navigation
fix(integration): stop re-geocoding unchanged addresses and cascading on 429 - #607
Merged
Merged
Conversation
… on 429 Firehose updates re-sent unchanged records and geocoded the same address on every update. Nominatim answered with 429, and each 429 fell through to a shorter address variant with another 1.1s wait, so ingest requests ran 5-10s and tripped the SlowAPIResponse alert. - Skip geocoding on update when the address is unchanged and the event already has coordinates - Return null on the first 429 instead of retrying shorter variants - Add a 5s timeout to the Nominatim request
PDS 0.4.5036 answers resolveHandle for a missing handle with error "HandleNotFound" instead of "InvalidRequest". isHandleAvailable only recognized the old form, so it threw on every available handle and account provisioning never created an atproto identity. Accept both forms so the check works on the current PDS and on older ones.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Firehose ingest requests to
POST /api/integration/eventswere taking 5 to 10 seconds. This kept tripping the slow API alert.The time was spent in Nominatim geocoding:
The processing duration metric did not show this, because its timer stops before the update or create step runs.
Change
Tests
Three new unit tests: unchanged address skips the geocoder, changed address still geocodes, and a 429 makes exactly one request. The first and third fail without this change. The full
event-integration.service.spec.tssuite passes (38 tests).Second fix: PDS handle check (unblocks e2e)
The e2e job started failing because the CI devnet pulls
ghcr.io/bluesky-social/pds:0.4, and that tag moved to PDS 0.4.5036 on 2026-09-30. That version answersresolveHandlefor a missing handle with errorHandleNotFound. Older versions answer withInvalidRequest.isHandleAvailableonly accepted the old form, so it threw on every free handle, and no test account got an atproto identity.isHandleAvailablenow accepts both forms. The production PDS is pinned to an older version and still returnsInvalidRequest, so this also keeps signup working after a future PDS upgrade.Checked locally with the CI compose stack and the same PDS image: the 6 suites that failed in CI pass (40 tests).