AI-180: Disable LangSmith network calls in samples tests via mock client#304
Open
AI-180: Disable LangSmith network calls in samples tests via mock client#304
Conversation
The langsmith_tracing tests previously instantiated `LangSmithPlugin()` with no client. Internally that constructs a real `langsmith.Client` which authenticates against `api.smith.langchain.com` using whatever `LANGSMITH_API_KEY` is in env, producing 401/403 warnings on every test run when a stale or invalid key was present. The samples tests don't assert on trace output, so the right fix is to bypass the network entirely. Added a `mock_ls_client` pytest fixture in `tests/langsmith_tracing/conftest.py` and wired it into `test_basic.py` and `test_chatbot.py` via `LangSmithPlugin(client=mock_ls_client)`. Verified empirically with `pytest-socket --disable-socket --allow-hosts=127.0.0.1,::1,localhost` and `LANGSMITH_API_KEY` set: zero non-localhost connections attempted during the test run. Drive-by: corrected a pre-existing inaccurate docstring on `test_chatbot_read_note` (the test saves a note first, then reads it back; the original elided the save phase). The underlying SDK behavior — `LangSmithInterceptor` force-sets `tracing_context(enabled=True)`, which overrides the standard `LANGSMITH_TRACING=false` disable mechanism — is filed separately as AI-183. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/langsmith_tracing/test_basic.pyandtest_chatbot.pypreviously instantiatedLangSmithPlugin()with no client. Internally that constructs a reallangsmith.Clientwhich authenticates againstapi.smith.langchain.comusing whateverLANGSMITH_API_KEYis in env, producing 401/403 warnings on every test run when a stale key was present.mock_ls_clientpytest fixture (in a newtests/langsmith_tracing/conftest.py) and wired it into both tests viaLangSmithPlugin(client=mock_ls_client). The mock is aMagicMockwith explicit.sessionand.tracing_queuestubs that the langsmith library accesses internally.pytest-socket --disable-socket --allow-hosts=127.0.0.1,::1,localhostandLANGSMITH_API_KEYset: zero non-localhost connections attempted, even with the env var present.test_chatbot_read_notethat didn't mention the save-first phase of the test.Related
LangSmithInterceptorforce-setstracing_context(enabled=True), which overridesLANGSMITH_TRACING=false. That's why the env-var disable mechanism couldn't be used here and a mock client is the correct fix in the samples.Test plan
uv run --group langsmith-tracing pytest tests/langsmith_tracing/ -v— 3/3 passuv run ruff check tests/langsmith_tracing/— cleanuv run ruff format --check tests/langsmith_tracing/— cleanpytest-socket --disable-socket --allow-hosts=127.0.0.1,::1,localhostwithLANGSMITH_API_KEYset