[SC-15741] Pluggable OIDC credentials backend for library auth

[jamadriz]

What and why?

This PR adds an enterprise security enhancement: pluggable OIDC credential storage for deployments that need tokens in an approved secret store (OS keychain, Vault, etc.) or memory-only sessions on shared Jupyter hosts — rather than the default local file cache.

The default remains ~/.validmind/credentials.json with user-only permissions, which continues to work well for individual development and notebook use. Organizations with stricter policies can opt in without forking the library.

This PR introduces a storage-only adapter. OIDC discovery, device flow, refresh, and Bearer headers remain owned by the SDK.

Before: OIDC tokens always persisted to ~/.validmind/credentials.json.

After: Default behavior unchanged. Deployments that need it can also:

• Pass credentials_backend= to vm.init()
• Set VM_OIDC_CREDENTIALS_BACKEND=module.path:ClassName (+ optional VM_OIDC_CREDENTIALS_BACKEND_KWARGS)
• Set VM_OIDC_NO_PERSIST=1 for memory-only storage (re-auth each process)
• Call clear_oidc_credentials() to wipe the active session cache

How to test

cd validmind-library
uv run python -m unittest tests.test_credentials_backend tests.test_credentials_store tests.test_api_client.TestAPIClientOIDC -v

Manual: implement a trivial backend, pass it to vm.init(..., credentials_backend=...), confirm tokens round-trip without creating ~/.validmind/credentials.json.

What needs special review?

• Adapter contract (get / put / delete keyed by credential_key) — sufficient for Vault/keychain integrations?
• Env-based backend loading (VM_OIDC_CREDENTIALS_BACKEND) for JupyterHub injection
• Whether docs/local/oidc_credentials_backend_adapter.md should stay in-repo or move to product docs

Dependencies, breaking changes, and deployment notes

• Builds on existing OIDC device-flow support in vm.init() (no backend changes required).
• Non-breaking: default remains file-backed ~/.validmind/credentials.json.
• New env vars: VM_OIDC_CREDENTIALS_BACKEND, VM_OIDC_CREDENTIALS_BACKEND_KWARGS, VM_OIDC_NO_PERSIST.

Release notes

Enhancement: OIDC library authentication supports an optional pluggable credentials backend for enterprise deployments. Store device-flow tokens in a custom secret store via credentials_backend or VM_OIDC_CREDENTIALS_BACKEND, or use memory-only storage with VM_OIDC_NO_PERSIST=1. Default local file caching is unchanged.

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

PR Summary

This pull request introduces a pluggable backend adapter for persisting OIDC credentials in the ValidMind library. The core change allows users to swap out the default filesystem storage (using ~/.validmind/credentials.json) with alternative storage solutions such as an OS keychain, corporate secret manager, or even memory-only storage for CI and shared environments.

Key functional changes include:

• The introduction of the OIDC credentials backend protocol which defines a simple storage interface with get, put, and delete methods. This protocol enables customers to implement their own secure token persistence mechanisms while reusing the existing device flow and token refresh logic.

• Multiple built-in backend implementations:

• FileCredentialsBackend: Continues to use the default file-based storage.
• MemoryCredentialsBackend: Provides an in-memory alternative suited for transient storage scenarios.
• NullCredentialsBackend: An alias for memory-only storage, simplifying configuration for non-persisting scenarios.

• Enhanced configuration and resolution logic that supports explicit backend specification as a parameter, a process-wide default, or via environment variables (e.g., VM_OIDC_CREDENTIALS_BACKEND, VM_OIDC_CREDENTIALS_BACKEND_KWARGS, and VM_OIDC_NO_PERSIST).

• Updates to the token obtaining and caching workflow in the API client. The new backend adapter is now threaded through token retrieval, caching, and clearing functions, ensuring consistency in how tokens are stored and removed.

• Comprehensive updates to tests to verify the functionality and integration of the new backend implementations. These tests cover token storage roundtrips, environment variable based backend resolution, and the proper handling of token cleanup.

Overall, these changes provide enhanced flexibility and security for deployments with different credential storage requirements, without altering the underlying OIDC device flow and refresh logic.

Test Suggestions

• Verify the backend resolution precedence by testing with explicit parameters, process-wide defaults, and environment variable configurations.
• Add tests to simulate error conditions, such as invalid backend specifications or failure during backend instantiation, to ensure appropriate exceptions are raised.
• Implement concurrent access tests to check the thread-safety of the memory and file backends.
• Create integration tests using a mock corporate secret manager to validate custom backend functionality in enterprise scenarios.