Repository navigation
refactor: generate the sync SDK from the async source with unasync - #189
Conversation
Pure git mv of each module that holds both an Async* class and its sync twin to its path under introspection_sdk/_async/, with no content change, so the next commits show as edits of the moved files. The tree does not import at this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
ea1fbf6 to
cd7ecd3
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
In each moved module, delete the sync classes and functions so the file holds the async source alone; the helper functions and constants stay where they were, and unasync copies them into the sync twin. The async docstrings carry the full text. Six modules get a sibling _<module>_common.py for what both twins must share as one object, which cannot live in the public module that imports the twins: auth (OAuthToken, AuthSession, SignInSupersededError, _EmailCodeAuthBase), _http (RawResponse), events (UnknownEventCounter, UNKNOWN_EVENT_SKIPS), runtimes (the hand-written RuntimeHandle / AsyncRuntimeHandle), conversations (ConversationExportParams) and streaming (_SseEvent, _SseAccumulator), plus the private helpers those classes call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restore each public module at its old path as a stub that re-exports the async twin from _async/ and the sync twin from _sync/ (and __all__), so every import path and public name stays as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sync.py Add scripts/gen_sync.py, which deletes introspection_sdk/_sync/, writes the sync twin of each _async/ module into it with unasync plus a token map and a few substitutions, then runs ruff over the folder, and check in its output. Every file there is generated. ruff's UP028 and C416 are off for _sync/, so the loops an `async for` leaves stay as written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run scripts/gen_sync.py in the Lint & Type Check job and as a pre-commit hook and fail on any diff or untracked file under introspection_sdk/_sync/, load scripts/griffe_reexports.py in the API compatibility check so re-exported classes are compared member by member, pin unasync in the dev extra (deptry DEP004 ignore scoped to it), lower the jscpd threshold to 16.3 for the new layout, and document the sync code generation in AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cd7ecd3 to
c5aebfe
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Same change as #186, rebuilt from
mainas five commits that git and GitHub can follow, so the review is readable commit by commit. Since #186, commit 2 has been rebuilt so that helper functions stay in the_async/module they came from. Only 6 modules keep a_<module>_common.py(see below), not 17.Commits
refactor: move the mixed sync/async modules under _async/: puregit mvof the 25 mixed modules tointrospection_sdk/_async/, no content change. GitHub shows every file as renamed with 0 changes; nothing to read. (Unchanged SHA.)refactor(_async): keep only the async twins: read this one closely. It deletes the sync classes from each moved module (+1,088 / −5,475 over 35 files). For 11 of the modules (tags, members, repositories, connectors, shares, tasks, files, connections, automations,_reads,proxy/_transport) the change is only deletions, plus the one-line module docstring (Re-exported by :mod:…``, which reads right in both twins) and the imports. It also renames_acp_post/ `_apost_token_form` to `_cp_post_async` / `_post_token_form_async` and makes `_http` sleep through a hand-written `_sleep` / `_sleep_async` pair in `_backoff.py`, so the generator needs no special rule for either. The helpers stay where they were, and unasync copies them into `_sync/`. Because of commit 1, every file shows as an edit of the old module.refactor: re-export both twins from the public modules: the stubs at the old paths that re-export both twins and__all__.build(unasync): generate the sync twins into _sync/ with scripts/gen_sync.py: readscripts/gen_sync.py(102 lines); skim_sync/. It is generated, and CI regenerates it and fails on any difference. Also adds the ruffper-file-ignoresfor_sync/(UP028,C416).ci: check the generated sync code and re-exported API in CI: CI steps, the griffe extension, jscpd threshold, AGENTS.md, pre-commit, deptry,uv.lock.Why 6
_commonmodules remainSome objects have to be one object that both twins use, not two copies. A twin can't import them from its public module, because that module imports the twin. These are the only things the 6 modules hold:
_auth_common.pyOAuthToken,AuthSession,SignInSupersededError,_EmailCodeAuthBaseisinstance/except)._EmailCodeAuthBaseis the base of the hand-writtenEmailCodeAuth/AsyncEmailCodeAuth._http_common.pyRawResponserunner_resources/_events_common.pyUnknownEventCounter,UNKNOWN_EVENT_SKIPSresources/_runtimes_common.pyRuntimeHandle,AsyncRuntimeHandleruntime_idis a property against a coroutine), and both generatedRuntimesclasses return them.runner_resources/_conversations_common.pyConversationExportParamsTypedDictthat both twins and both protocols take._streaming_common.py_SseEvent,_SseAccumulatorBesides those classes, the modules keep only the private helpers the classes themselves call:
_jwt_claims,_claim,_resolve_base_api_url,_utcnow,SessionListener,_is_uuidand the events_logger._is_rejection,_rejectedand_not_signed_inare used only by the hand-writtenEmailCodeAuthtwins, so they moved into the publicauth.pynext to them. Every other helper and constant now has an async copy and a generated sync copy; the public modules re-export the async copy under the old name. Private names that tests import (_http._clean_params,files._materialise_upload, …) are still re-exported.Commits 1 to 4 were made with
--no-verifyand don't import or pass tests on their own (commit 1 leaves the public modules missing until commit 3). Commit 5 is the full tree. On it I ranprek run --all-files(every hook passes, includingtyandgen-sync),ruff format --check,ruff check,ty check,deptry,pytest(774 passed, coverage 92.4%),scripts/gen_sync.py+git diff --exit-code,griffe check --against v0.29.0 -e scripts/griffe_reexports.py(exit 0, and exit 0 against v0.29.1) and jscpd (16.22%). Merge with squash as usual. This replaces #186; close that one once this is merged.Coverage note: total coverage is 92.36%, against 93.94% on the previous head and 93.6% on main. The
fail_under = 80ratchet holds and no test changed. The drop comes from the copied helpers: a test that calls a helper directly through its public name (parse_lookback,decode_arrow_page,_materialise_upload, …) covers the async copy, while the sync copy is covered only through the sync methods that call it. The same applies the other way round.The description and evidence below are from #186 and still apply to this tree, except where a section says it was updated.
Every class or function with an
Async*twin is now written once, as async code underintrospection_sdk/_async/.scripts/gen_sync.pygenerates the sync twin intointrospection_sdk/_sync/with unasync, pinned tounasync==0.6.0in thedevextra. httpx/httpcore and urllib3 work the same way.Layout
introspection_sdk/_async/<path>.py: the hand-written async twins, the single source.introspection_sdk/_sync/<path>.py: generated. Each file opens with a# Generated by scripts/gen_sync.py from … Do not editheader.introspection_sdk/<path>.py(the public module): re-exports both twins (and__all__), so every import path and public name stays as it was. The few objects both twins must share as one object live in a sibling_<module>_common.py(6 modules, see above). Private names that tests import from the public modules (_http._HttpClient,_clean_params,files._materialise_upload,streaming._parse_sse, …) are still re-exported._http, auth (token functions,CredentialProvider), streaming,proxy/{_transport,httpx,httpx2},resources/{connectors,experiments,members,recipes,repositories,runtimes,tags},runner_resources/{_reads,automations,connections,conversations,events,files,metrics,shares,tasks}. That is 55 sync classes and functions, about 2,800 generated lines.The generator (
scripts/gen_sync.py, updated for c5aebfe)asyncandawait.Async*and_Async*lose the prefix, andasync_*and*_asynclose the affix. An explicitTOKENSmap covers the rest:AsyncClient→Client,AsyncBaseTransport→BaseTransport,AsyncHTTPTransport→HTTPTransport,AsyncByteStream→SyncByteStream,aclose→close,aiter_bytes/aiter_lines/aiter_raw/aread,handle_async_request→handle_request,_async→_sync, plus unasync's defaults (AsyncIterator→Iterator,__aenter__→__enter__, …).SUBSTITUTIONShandles what isn't a single token:Awaitable[X]becomesX, prose cross-references like:class:`AsyncPager`lose the prefix,AsynchronousbecomesSynchronous, and an# unasync: async-only…# unasync: endfence marks a block the sync twin doesn't have. Today that fence covers the async client's__aenter__/__aexit__(the sync client is no context manager) and 35 async docstrings that say something the sync one doesn't.introspection_sdk/_sync/and generates into it in place. Then ruff runs once over the folder with the repo config:ruff check --fix --select I,F401(sorts imports, drops the ones only the async code used; no--unsafe-fixes), thenruff format. Thefor x in y: yield xand[x for x in y]thatasync forleaves stay literal;pyproject.tomlturnsUP028andC416off forintrospection_sdk/_sync/**.--checkmode: theLint & Type Checkjob and thegen-syncpre-commit hook run the script, then fail ongit diff --exit-codeor any untracked file underintrospection_sdk/_sync.unasync, with a reason; the import is a dev-only generator.Not generated (hand-written, with reason)
pagination.py:Pager/AsyncPager(+cursor_paginatepair)Pagerdelegates attributes to the first page (__getattr__),AsyncPageris awaitable (__await__). The members differ, not just the tokens.resumable.py:stream_resumable/stream_resumable_asynctests/rest/test_resumable.pypatchesintrospection_sdk.resumable._retry_delay. A generated_sync/resumable.pywould look the name up in its own globals, so the patch would miss. Tests may not change.auth.py:EmailCodeAuth/AsyncEmailCodeAuththreadinglocks (with self._state:) that the async twin, on one event loop, doesn't take.resources/annotations.py:Annotations/AsyncAnnotationslist()resolves email filters eagerly in sync (errors raise at call time) and lazily on the first fetch in async (a synclist()can't await). Generating it would change sync behaviour.resources/runtimes.py:RuntimeHandle/AsyncRuntimeHandleruntime_idis a property on one and a coroutine on the other.Runtimes/AsyncRuntimesare generated.proxy/httpx.pyvsproxy/httpx2.pyPublic API
griffe check (
uv run griffe check introspection_sdk --search . --against v0.29.0 -e scripts/griffe_reexports.py, asapi-compat.ymlnow runs it): exit 0, no breakages, and noapi-breaklabel or!needed.seen_pathsguard before reaching the members (_type_based_yield→_alias_incompatibilities→_type_based_yieldon the same old path). So every moved class passes without its members being compared. I checked this: changing a default in_async/runner_resources/shares.pystill gave exit 0 against main. From the next release on, both sides are re-exports and the check catches changes again; I verified that by running--against HEADwith the same injected change, which griffe then reports. AGENTS.md now notes this. To cover this PR, I applied griffe's own breakage rules to the resolved targets of every public object (resolving aliases on both sides, with a freshseen_pathsper object). Againstv0.29.0and againstmain(86e42ba): 0 breakages. The same harness reports the injected default change, so it isn't vacuous.griffe dump comparison (public objects resolved through aliases; 64 modules, 514 classes, 962 functions, 2,686 attributes compared, against both
mainandv0.29.0):UnknownEventCounter, which logs through the shared_logger(same logger name) instead of the module-levellogger. Since c5aebfe, four statements differ from main by construction: twofor chunk in …: yield chunkloops where main hasyield from, one[table for table in self]where main haslist(self), and_sleep(delay)where main hastime.sleep(delay). Docstrings are unchanged, and the private_async/and_sync/module docstrings now read the same in both trees.Async twin of :meth:`Members.update`., or that had no docstring. They now carry the full text of the sync docstring. The 47 async docstrings with real content are byte-identical: 12 because they map to the sync text by prose substitution, 35 through the async-only fence. The 44 changed docstrings:auth.async_authorization_code_tokenauth.async_email_code_tokenauth.async_refresh_access_tokenauth.async_revoke_sessionauth.async_send_email_codeauth.async_service_account_tokenauth.async_token_exchangeauth.AsyncCredentialProviderauth.AsyncCredentialProvider.authorizationauth.AsyncCredentialProvider.refresh_after_unauthorizedresources.connectors.AsyncConnectorsresources.members.AsyncMembersresources.members.AsyncMembers.createresources.members.AsyncMembers.updateresources.recipes.AsyncRecipesresources.repositories.AsyncRepositoriesresources.repositories.AsyncRepositories.commitresources.repositories.AsyncRepositories.commitsresources.repositories.AsyncRepositories.listresources.repositories.AsyncRepositoryContentsresources.repositories.AsyncRepositoryMergesresources.repositories.AsyncRepositoryMerges.createrunner_resources.automations.AsyncAutomationsrunner_resources.automations.AsyncAutomations.createrunner_resources.automations.AsyncAutomations.deleterunner_resources.automations.AsyncAutomations.getrunner_resources.automations.AsyncAutomations.triggerrunner_resources.automations.AsyncAutomations.updaterunner_resources.connections.AsyncAppConnectionsrunner_resources.connections.AsyncAppConnections.createrunner_resources.connections.AsyncAppConnections.deleterunner_resources.connections.AsyncAppConnections.getrunner_resources.connections.AsyncAppConnections.listrunner_resources.connections.AsyncRunnerAppConnectionsrunner_resources.connections.AsyncRunnerAppConnections.createrunner_resources.conversations.AsyncConversations.export_arrowrunner_resources.conversations.AsyncConversations.export_jsonrunner_resources.conversations.AsyncConversations.export_streamrunner_resources.conversations.AsyncConversations.export_trajectoryrunner_resources.events.AsyncEventsrunner_resources.events.AsyncEvents.getrunner_resources.metrics.AsyncMetricsrunner_resources.shares.AsyncShares.updatestreaming.parse_ag_ui_events_asyncag_ui.coredon't resolve on either side (external package, unchanged).Runtime-visible change outside griffe: the moved classes'
__module__is nowintrospection_sdk._sync.…/introspection_sdk._async.…(for exampleintrospection_sdk.Runner.__module__). Import paths,isinstanceand pickling-by-reference through the public paths are unaffected. The events module logger keeps its name (introspection_sdk.runner_resources.events).Duplicate code (jscpd)
introspection_sdk/_sync/**goes in.jscpd.jsonignore; it repeats_async/by construction._commonmodules)_commonmodules)thresholdgoes from 29.3 to 16.3 (updated for cd7ecd3).Checks run locally
uv run pytest tests/: 774 passed, no test edits. Coverage 92.4% on c5aebfe (92.36% on cd7ecd3) (main 93.6%; see the coverage note above);--cov-fail-under=80holds. The fenced async docstrings count as statements, which adds 3 uncovered lines in async methods no test calls.ruff format --check,ruff check(incl. the complexity ratchet),ty check(pinned),mypy,deptry,jscpd(exit 0),actionlint,check_examples.py,scripts/gen_sync.py+git diff --exit-code: all pass.scripts/check_api_contract.py(scheduled, not a PR gate): output is byte-identical to main's.prek run --all-files: every hook passes on c5aebfe (and on cd7ecd3), the localtyhook included.API check: re-exported classes
griffe checkpassed vacuously for every class this PR moves behind a re-export:_type_based_yieldadds the old object's path toseen_paths, goes into_alias_incompatibilities, and recurses into the alias's target with that same old path, which returns at once. griffe has no CLI option for this, and griffe 2.4.0 (latest) has the samediff.py, so an upgrade doesn't help.The fix stays on the plain CLI and uses griffe's extension API instead of a custom checker:
scripts/griffe_reexports.pyis a ~20-linegriffe.Extensionwhoseon_packagehook replaces each public alias whose target lives inintrospection_sdkwith the target object. It runs on the old release and the new code alike (griffe loads both with the same extensions), so both sides hold real objects and griffe's own breakage rules compare them.api-compat.ymladds-e scripts/griffe_reexports.py; theapi-breaklabel and!title override is unchanged. AGENTS.md documents the flag.pyproject.tomlwidens the scoped deptry DEP004 ignore togriffe(dev-only import inscripts/).Evidence (the breaking change was applied locally only, never committed): remove
namefromAsyncRecipes.listin_async/resources/recipes.pyand regenerate_sync/.Clean HEAD (cd7ecd3; also exit 0 on c5aebfe):
With the breaking change, the old command vs the new one:
Re-run on cd7ecd3 and on c5aebfe with the same change in
_async/only (no regeneration):introspection_sdk/_async/resources/recipes.py:25: AsyncRecipes.list(name): Parameter was removed, exit 1.Docs
AGENTS.md gains a "Sync code generation" section (workflow, layout, token map, fence, the hand-written list, adding a twin). The "Duplicate code" section now mentions the
_sync/exclusion, and "Public API compatibility" describes the griffe extension below.🤖 Generated with Claude Code