Repository navigation
refactor: generate the sync SDK from the async source with unasync - #186
Closed
marcusschiesser wants to merge 3 commits into
Closed
marcusschiesser wants to merge 3 commits into
marcusschiesser wants to merge 3 commits into
Conversation
Every class or function with an Async* twin is now written once, as async code under introspection_sdk/_async/, and scripts/unasync.py generates the sync twin into introspection_sdk/_sync/ with unasync (pinned in the dev extra), as httpx/httpcore and urllib3 do. The public modules re-export both twins, so every import path and public name is unchanged; code both twins share moves to sibling _<module>_common.py modules. The generated files carry a "do not edit" header, and CI and a pre-commit hook regenerate them and fail on any diff. Code that genuinely differs between sync and async stays hand-written: Pager/AsyncPager, stream_resumable, EmailCodeAuth, Annotations and RuntimeHandle. The generated tree is excluded from jscpd, which drops the duplicated-lines share of introspection_sdk/ from 29.2% to 16.63%; the threshold goes from 29.3 to 16.7. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
griffe check skips the members of a class that moved behind a re-export: _type_based_yield records the old path in seen_paths, then recurses into the alias's target with that same path and returns. Load a griffe extension that swaps each public in-package alias for its target before the check, so the breakage rules apply to the moved classes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Contributor
Author
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every class or function with an
Async*twin is now written once, as async code underintrospection_sdk/_async/.scripts/unasync.pygenerates the sync twin intointrospection_sdk/_sync/with unasync, pinned tounasync==0.6.0in thedevextra. httpx/httpcore and urllib3 work the same way.Layout
introspection_sdk/_async/<path>.py: the hand-written async twins, the single source.introspection_sdk/_sync/<path>.py: generated. Each file opens with a# Generated by scripts/unasync.py from … Do not editheader.introspection_sdk/<path>.py(the public module): re-exports both twins (and__all__), so every import path and public name stays as it was. Code both twins share moves to a sibling_<module>_common.py, because a twin can't import its own public module, which imports the twin. Private names that tests import from the public modules (_http._HttpClient,_clean_params,files._materialise_upload,streaming._parse_sse, …) are still re-exported._http, auth (token functions,CredentialProvider), streaming,proxy/{_transport,httpx,httpx2},resources/{connectors,experiments,members,recipes,repositories,runtimes,tags},runner_resources/{_reads,automations,connections,conversations,events,files,metrics,shares,tasks}. That is 55 sync classes and functions, about 2,800 generated lines.The generator (
scripts/unasync.py)asyncandawait.Async*and_Async*lose the prefix, andasync_*and*_asynclose the affix. An explicitTOKENSmap covers the rest:AsyncClient→Client,AsyncBaseTransport→BaseTransport,AsyncHTTPTransport→HTTPTransport,AsyncByteStream→SyncByteStream,aclose→close,aiter_bytes/aiter_lines/aiter_raw/aread,handle_async_request→handle_request,_async→_sync,_acp_post→_cp_post,_apost_token_form→_post_token_form, plus unasync's defaults (AsyncIterator→Iterator,__aenter__→__enter__, …).SUBSTITUTIONShandles what isn't a single token:import asyncio/asyncio.sleep(becometime/time.sleep(,Awaitable[X]becomesX, prose cross-references like:class:`AsyncPager`lose the prefix,AsynchronousbecomesSynchronous, and an# unasync: async-only…# unasync: endfence marks a block the sync twin doesn't have. Today that fence covers the async client's__aenter__/__aexit__(the sync client is no context manager) and 35 async docstrings that say something the sync one doesn't.for x in y: yield xand[x for x in y]left byasync forintoyield fromandlist(). Then it formats.--checkregenerates in memory and fails with a diff. It's a new step in theLint & Type Checkjob and a newunasyncpre-commit hook.sys.pathbeforeimport unasync. deptry gets a DEP004 ignore scoped tounasync, with a reason; the import is a dev-only generator.Not generated (hand-written, with reason)
pagination.py:Pager/AsyncPager(+cursor_paginatepair)Pagerdelegates attributes to the first page (__getattr__),AsyncPageris awaitable (__await__). The members differ, not just the tokens.resumable.py:stream_resumable/stream_resumable_asynctests/rest/test_resumable.pypatchesintrospection_sdk.resumable._retry_delay. A generated_sync/resumable.pywould look the name up in its own globals, so the patch would miss. Tests may not change.auth.py:EmailCodeAuth/AsyncEmailCodeAuththreadinglocks (with self._state:) that the async twin, on one event loop, doesn't take.resources/annotations.py:Annotations/AsyncAnnotationslist()resolves email filters eagerly in sync (errors raise at call time) and lazily on the first fetch in async (a synclist()can't await). Generating it would change sync behaviour.resources/runtimes.py:RuntimeHandle/AsyncRuntimeHandleruntime_idis a property on one and a coroutine on the other.Runtimes/AsyncRuntimesare generated.proxy/httpx.pyvsproxy/httpx2.pyPublic API
griffe check (
uv run griffe check introspection_sdk --search . --against v0.29.0 -e scripts/griffe_reexports.py, asapi-compat.ymlnow runs it): exit 0, no breakages, and noapi-breaklabel or!needed.seen_pathsguard before reaching the members (_type_based_yield→_alias_incompatibilities→_type_based_yieldon the same old path). So every moved class passes without its members being compared. I checked this: changing a default in_async/runner_resources/shares.pystill gave exit 0 against main. From the next release on, both sides are re-exports and the check catches changes again; I verified that by running--against HEADwith the same injected change, which griffe then reports. AGENTS.md now notes this. To cover this PR, I applied griffe's own breakage rules to the resolved targets of every public object (resolving aliases on both sides, with a freshseen_pathsper object). Againstv0.29.0and againstmain(86e42ba): 0 breakages. The same harness reports the injected default change, so it isn't vacuous.griffe dump comparison (public objects resolved through aliases; 64 modules, 514 classes, 962 functions, 2,686 attributes compared, against both
mainandv0.29.0):Async twin of :meth:`Members.update`., or that had no docstring. They now carry the full text of the sync docstring. The 47 async docstrings with real content are byte-identical: 12 because they map to the sync text by prose substitution, 35 through the async-only fence. The 44 changed docstrings:auth.async_authorization_code_tokenauth.async_email_code_tokenauth.async_refresh_access_tokenauth.async_revoke_sessionauth.async_send_email_codeauth.async_service_account_tokenauth.async_token_exchangeauth.AsyncCredentialProviderauth.AsyncCredentialProvider.authorizationauth.AsyncCredentialProvider.refresh_after_unauthorizedresources.connectors.AsyncConnectorsresources.members.AsyncMembersresources.members.AsyncMembers.createresources.members.AsyncMembers.updateresources.recipes.AsyncRecipesresources.repositories.AsyncRepositoriesresources.repositories.AsyncRepositories.commitresources.repositories.AsyncRepositories.commitsresources.repositories.AsyncRepositories.listresources.repositories.AsyncRepositoryContentsresources.repositories.AsyncRepositoryMergesresources.repositories.AsyncRepositoryMerges.createrunner_resources.automations.AsyncAutomationsrunner_resources.automations.AsyncAutomations.createrunner_resources.automations.AsyncAutomations.deleterunner_resources.automations.AsyncAutomations.getrunner_resources.automations.AsyncAutomations.triggerrunner_resources.automations.AsyncAutomations.updaterunner_resources.connections.AsyncAppConnectionsrunner_resources.connections.AsyncAppConnections.createrunner_resources.connections.AsyncAppConnections.deleterunner_resources.connections.AsyncAppConnections.getrunner_resources.connections.AsyncAppConnections.listrunner_resources.connections.AsyncRunnerAppConnectionsrunner_resources.connections.AsyncRunnerAppConnections.createrunner_resources.conversations.AsyncConversations.export_arrowrunner_resources.conversations.AsyncConversations.export_jsonrunner_resources.conversations.AsyncConversations.export_streamrunner_resources.conversations.AsyncConversations.export_trajectoryrunner_resources.events.AsyncEventsrunner_resources.events.AsyncEvents.getrunner_resources.metrics.AsyncMetricsrunner_resources.shares.AsyncShares.updatestreaming.parse_ag_ui_events_asyncag_ui.coredon't resolve on either side (external package, unchanged).Runtime-visible change outside griffe: the moved classes'
__module__is nowintrospection_sdk._sync.…/introspection_sdk._async.…(for exampleintrospection_sdk.Runner.__module__). Import paths,isinstanceand pickling-by-reference through the public paths are unaffected. The events module logger keeps its name (introspection_sdk.runner_resources.events).Duplicate code (jscpd)
introspection_sdk/_sync/**goes in.jscpd.jsonignore; it repeats_async/by construction.thresholdgoes from 29.3 to 16.7.Checks run locally
uv run pytest tests/: 774 passed, no test edits. Coverage 93.9% (main 93.6%);--cov-fail-under=80holds. The fenced async docstrings count as statements, which adds 3 uncovered lines in async methods no test calls.ruff format --check,ruff check(incl. the complexity ratchet),ty check(pinned),mypy,deptry,jscpd(exit 0),actionlint,check_examples.py,scripts/unasync.py --check: all pass.scripts/check_api_contract.py(scheduled, not a PR gate): output is byte-identical to main's.prek run --all-files: every hook passes except the local unpinneduvx ty. That gives the same 37 diagnostics on main (they move along with the code), while CI's pinned ty passes.API check: re-exported classes
griffe checkpassed vacuously for every class this PR moves behind a re-export:_type_based_yieldadds the old object's path toseen_paths, goes into_alias_incompatibilities, and recurses into the alias's target with that same old path, which returns at once. griffe has no CLI option for this, and griffe 2.4.0 (latest) has the samediff.py, so an upgrade doesn't help.The fix stays on the plain CLI and uses griffe's extension API instead of a custom checker:
scripts/griffe_reexports.pyis a ~20-linegriffe.Extensionwhoseon_packagehook replaces each public alias whose target lives inintrospection_sdkwith the target object. It runs on the old release and the new code alike (griffe loads both with the same extensions), so both sides hold real objects and griffe's own breakage rules compare them.api-compat.ymladds-e scripts/griffe_reexports.py; theapi-breaklabel and!title override is unchanged. AGENTS.md documents the flag.pyproject.tomlwidens the scoped deptry DEP004 ignore togriffe(dev-only import inscripts/).Evidence (the breaking change was applied locally only, never committed): remove
namefromAsyncRecipes.listin_async/resources/recipes.pyand regenerate_sync/.Clean HEAD (c637ad2):
With the breaking change, the old command vs the new one:
Docs
AGENTS.md gains a "Sync code generation" section (workflow, layout, token map, fence, the hand-written list, adding a twin). The "Duplicate code" section now mentions the
_sync/exclusion, and "Public API compatibility" describes the griffe extension below.🤖 Generated with Claude Code