feat(auth): OIDC device flow & API audience for library (sc-15741)

[jamadriz]

Pull Request Description

Backend PR - https://github.com/validmind/backend/pull/3075

What and why?

This PR adds OAuth 2.0 / OIDC device authorization flow support to the ValidMind Python library so notebooks can authenticate with a Bearer token instead of only API keys.

Behavior:

• vm.init(..., issuer=..., client_id=..., scope=..., audience=...) runs RFC 8628 device login, caches tokens under ~/.validmind/credentials.json, and sends Authorization: Bearer ... to the tracking API.
• audience (or env VM_OIDC_AUDIENCE) is passed to the device authorize and token endpoints so Auth0 (and similar IdPs) can issue RS256 API access tokens for the resource identifier that matches the backend api_audience.
• normalize_issuer, normalize_client_id, and normalize_audience strip common wrapping quotes from copied .env / notebook values.
• Credential cache keys are scoped by issuer, client id, and audience when audience is set so tokens for different APIs do not collide.
• New modules: validmind/oidc_device.py, validmind/credentials_store.py. Auth failures raise ValidMindAuthError (extends existing error hierarchy).
• Tutorial notebooks and OIDC plan doc updated with setup guidance.

Before: Library authentication required api_key / api_secret only.
After: Either API keys or OIDC device flow parameters can be used (mutually exclusive).

How to test

1. pip install -e . from this repo; restart the Jupyter kernel.
2. In Auth0: Native app with Device Code grant; API with Identifier equal to backend api_audience; authorize the app for that API.
3. vm.init( api_host='', model='<your model id>', document="documentation", issuer='', client_id='', audience='', ) (empty strings prevent env API keys from mixing with OIDC during local tests).
4. Run unit tests:

pytest tests/test_credentials_store.py tests/test_oidc_device.py tests/test_api_client.py::TestAPIClientOIDC -q

What needs special review?

• OIDC parameter naming and env var (VM_OIDC_AUDIENCE) vs platform docs.
• Notebook edits (large diff may include cell outputs — consider clearing outputs before merge if policy requires).

Dependencies, breaking changes, and deployment notes

• Pairs with backend PR that accepts Bearer tokens on tracking routes and verifies OIDC JWTs (JWKS / HS256 as configured).
• No breaking change for existing API-key users.
• After release, PyPI consumers need a version that includes audience on vm.init for RS256 API-token flows.

Release notes

Enhancement: Optional OAuth device login for vm.init via issuer, client_id, optional scope, and optional audience (Auth0 API Identifier). Environment variable VM_OIDC_AUDIENCE supported. Tokens cached under ~/.validmind/credentials.json.

Suggested label: enhancement, environment-variables

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut (sc-15741)
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ jamadriz
❌ mdeyell-valid-mind
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[cachafla]

Why not derive api_url from api_host?

No need to offer a new argument to a user if they both accomplish the same.

[jamadriz]

sure, the ticket actually had api_url specified in the story hence why it was added

[jamadriz]

right now they are both the same (oidc device flow will still work passing api_host and not api_url), api_url seems better naming though, since we were using api_host as actually being the tracking url, with the full tracking namespace route. I suggest we leave both, and we deprecate api_host on followup PRs

[cachafla]

Looking good 👍 I'd suggest an accompanying notebook to allow users to test this without changing an existing notebook.

Maybe we can copy quickstart_model_documentation.ipynb and create a new one called quickstart_model_documentation_oidc_device_flow.ipynb?

Only difference in content is an explanation at the beginning about what parameters to pass to vm.init() and how to complete the flow.

[mdeyell-valid-mind]

I'll dig into this more tomorrow but when trying to use an entra token the backend gives me the error below. I think the token is generated correctly since it gives me a code to put into a web browser.

2026-05-04T16:15:31.562445 [info] JWT validation error for m$: InvalidSignatureError filename=auth_oidc.py func_name=verify_auth_token lineno=139 method=GET module=auth.auth_oidc path=/api/v1/tracking/ping request_id=4af3cdbd-fc99-4637-826c-799298d51353

[jamadriz]

Looking good 👍 I'd suggest an accompanying notebook to allow users to test this without changing an existing notebook.

Maybe we can copy quickstart_model_documentation.ipynb and create a new one called quickstart_model_documentation_oidc_device_flow.ipynb?

Only difference in content is an explanation at the beginning about what parameters to pass to vm.init() and how to complete the flow.

done

[cachafla]

No need to provide a demo notebook for this one. The quickstart is good enough.

[cachafla]

No need to refer to a separate file here:

...see [OIDC device flow release notes](../../docs/oidc-device-flow-release-notes.md).

Better to prevent this kind of linking since that will force us to carry 2 files around: this one and the markdown. Better to link it to a ValidMind docs page if needed.

[cachafla]

This should also be removed in the #### Configure vm.init() for OIDC section.

[cachafla]

    issuer="https://login.microsoftonline.com/<tenant-id>/v2.0",  # your IdP issuer (OpenID discovery)

Why not use ValidMind's prod URL so it can be tested without looking up the prod login authority?

[jamadriz]

@cachafla I didn’t see valimind url in any notebooks vm.init so I assumed we don’t want to include this here. I will add issuer per your request (but leave client id empty, maybe we want to add this somewhere in the app instead)

[mdeyell-valid-mind]

To get ping to work with Entra I had to change this

+def _is_entra_issuer(issuer: str) -> bool:
+    return "login.microsoftonline.com" in issuer.lower()
+
+
+def _select_oidc_bearer_token(entry: Dict[str, Any]) -> str:
+    if _is_entra_issuer(entry.get("issuer", "")) and entry.get("id_token"):
+        return entry["id_token"]
+    return entry["access_token"]
+
+
 def init(
     api_key: Optional[str] = None,
     api_secret: Optional[str] = None,
@@ -380,7 +390,7 @@ def init(
         entry = _obtain_oidc_tokens(
             issuer, client_id, scope_val, audience=oidc_audience_opt
         )
-        _access_token = entry["access_token"]
+        _access_token = _select_oidc_bearer_token(entry)
         _oidc_login_context = {
             "issuer": entry["issuer"],
             "client_id": entry["client_id"],

I also verified that I can call log_input with Entra

[mdeyell-valid-mind]

To get ping to work with Entra I had to change this

+def _is_entra_issuer(issuer: str) -> bool:
+    return "login.microsoftonline.com" in issuer.lower()
+
+
+def _select_oidc_bearer_token(entry: Dict[str, Any]) -> str:
+    if _is_entra_issuer(entry.get("issuer", "")) and entry.get("id_token"):
+        return entry["id_token"]
+    return entry["access_token"]
+
+
 def init(
     api_key: Optional[str] = None,
     api_secret: Optional[str] = None,
@@ -380,7 +390,7 @@ def init(
         entry = _obtain_oidc_tokens(
             issuer, client_id, scope_val, audience=oidc_audience_opt
         )
-        _access_token = entry["access_token"]
+        _access_token = _select_oidc_bearer_token(entry)
         _oidc_login_context = {
             "issuer": entry["issuer"],
             "client_id": entry["client_id"],

I also verified that I can call log_input with Entra

Unit tests

    @patch("validmind.api_client._ping")
    @patch("validmind.api_client._obtain_oidc_tokens")
    def test_init_entra_oidc_uses_id_token(self, mock_obtain, mock_ping):
        mock_obtain.return_value = {
            "issuer": "https://login.microsoftonline.com/tenant-id/v2.0",
            "client_id": "cid",
            "access_token": "access-token",
            "expires_at": "2099-01-01T00:00:00+00:00",
            "refresh_token": None,
            "id_token": "id-token",
        }
        api_client.init(
            model="model-cuid",
            api_host="http://localhost/track/",
            api_key="",
            api_secret="",
            issuer="https://login.microsoftonline.com/tenant-id/v2.0",
            client_id="cid",
            document="documentation",
        )
        headers = api_client._get_api_headers()
        self.assertEqual(headers["Authorization"], "Bearer id-token")

    @patch("validmind.api_client._ping")
    @patch("validmind.api_client._obtain_oidc_tokens")
    def test_init_non_entra_oidc_prefers_access_token(self, mock_obtain, mock_ping):
        mock_obtain.return_value = {
            "issuer": "https://issuer.example.com/",
            "client_id": "cid",
            "access_token": "access-token",
            "expires_at": "2099-01-01T00:00:00+00:00",
            "refresh_token": None,
            "id_token": "id-token",
        }
        api_client.init(
            model="model-cuid",
            api_host="http://localhost/track/",
            api_key="",
            api_secret="",
            issuer="https://issuer.example.com/",
            client_id="cid",
            document="documentation",
        )
        headers = api_client._get_api_headers()
        self.assertEqual(headers["Authorization"], "Bearer access-token")

[jamadriz]

@mdeyell-valid-mind with these changes for Entra, did you have a chance to ensure it keeps working with Auth0?
can you push these to the branch?
thanks

[mdeyell-valid-mind]

@mdeyell-valid-mind with these changes for Entra, did you have a chance to ensure it keeps working with Auth0? can you push these to the branch? thanks

I haven't ran this for auth0 yet but i'll check

[mdeyell-valid-mind]

works on my machine

[mdeyell-valid-mind]

I tested with entra, auth0, and api keys

[cachafla]

Approving now but the quickstart notebook still needs to be updated to not refer to a separate file (docs/oidc-device-flow-release-notes.md) because we need to ensure this notebook is portable and does not have any extra dependencies.

[github-actions]

PR Summary

This pull request introduces a comprehensive set of enhancements for OIDC (OpenID Connect) device flow authentication and overall API client token management in the ValidMind library. The changes are reflected across several modules:

1. OIDC Device Flow Integration:

   • The new OIDC pathways have been integrated within the API client. The logic to obtain tokens via the device authorization flow (RFC 8628) has been added along with token refresh support and a mechanism to cache tokens in a secure credentials store (stored under ~/.validmind/credentials.json).
   • The code distinguishes between API-key based authentication and OIDC-based authentication. When OIDC is used, the client selects the correct bearer token (preferring the id_token when the issuer is from Microsoft Entra) and sets the appropriate HTTP header.
   • New test cases under tests/test_api_client.py and tests/test_oidc_device.py validate the behavior under various scenarios such as mixing API-key with OIDC parameters, missing required parameters, proper header assignment (Bearer token), and dynamic interval adjustments during token polling.

2. Documentation and Notebook Updates:

   • A new release note document (docs/oidc-device-flow-release-notes.md) provides detailed instructions on how the OIDC device flow works. A quickstart notebook (notebooks/quickstart/quickstart_model_documentation_oidc_device_flow.ipynb) now walks users through model registration, dataset initialization, and running the full suite of tests in OIDC mode.
   • The documentation explains IdP configuration requirements (discovery endpoints, audience, scopes), environment variable overrides, and operational considerations (e.g. token caching and refreshing).

3. Credential Store Improvements:

   • The credentials store (validmind/credentials_store.py) now features utility functions for normalizing issuer, client id, and audience, ensuring consistent key generation for token caching. Tests in tests/test_credentials_store.py validate proper behavior in caching, expiry, and file writing (with atomic writes and correct file permissions).

4. Additional Bugfixes and Code Style Updates:

   • Adjustments are made around treatment of pandas categorical types to use isinstance(dtype, pd.CategoricalDtype) rather than deprecated checks, preserving data type information for downstream processes.
   • Minor improvements in async session invalidation and consistent error messaging have been incorporated.

Overall, the PR refines the authentication flow for scenarios where API key and OIDC mode must not be mixed, ensuring security and clarity while providing an improved developer experience through enhanced documentation and comprehensive tests.

Test Suggestions

• Run the full test suite (unit and integration tests) locally to verify both API-key and OIDC flows.
• Manually trigger the OIDC device flow in a notebook environment to ensure the QR code displays correctly and instructions are clear.
• Simulate an expired token scenario and verify that the token refresh mechanism correctly obtains and caches new tokens.
• Test the behavior when both API keys and OIDC parameters are provided to ensure that the proper error is raised.
• Verify that the credentials file is created with proper permissions and that corrupted content raises an appropriate exception.