UN-4009 [MISC] Generate and commit the API deployment OpenAPI spec in-repo - #2237
Open
chandrasekharan-zipstack wants to merge 7 commits into
Open
UN-4009 [MISC] Generate and commit the API deployment OpenAPI spec in-repo#2237chandrasekharan-zipstack wants to merge 7 commits into
chandrasekharan-zipstack wants to merge 7 commits into
Conversation
The published Python clients and their generated SDKs are built from a spec of the deployment execute/status endpoints, which until now was produced by a script living outside this repo — so a route or serializer change here could silently invalidate it. The schema annotation for DeploymentExecution now lives next to the view, and `manage.py generate_docstudio_spec` writes specs/docstudio-oss.json. A unit test regenerates and compares, so drift fails in this repo's existing CI tier rather than in a client repo, with no database or extra CI job needed. The generated spec is unchanged from what the external script produced, apart from a root `tags` array — clients had nowhere to read group descriptions from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
The committed spec is what published clients are generated from, so the places where it disagreed with the server are places every SDK inherits. - Declare the bearer scheme the endpoints enforce. DRF's unset authentication default was being introspected as a decision and published session and basic auth, which these endpoints do not accept. - Declare the failures a caller has to handle (400/401/403/404/409/429) and describe the 406, so a generated client can branch on them. - Derive the response model from the serializer that builds the response, which drops `workflow_id` -- a field no code path produces. - Stop shadowing `files`: the real field carries the binary annotation, so a change to it now moves the spec. - Drop the MCP operations. MCP speaks JSON-RPC over one POST, so it had no REST shape to describe and was published with guessed responses, no security, and an internal docstring as its description. - Say in the shipped text that a status read is one-shot, and that documents may be supplied as files or presigned URLs. The gate had the same blind spots. It now resolves the real mount instead of comparing against a hand-written copy of it, fails when the generator reports a diagnostic instead of certifying its guess, and asserts the auth scheme and error statuses. Verified by mutation: moving the mount, adding a response field, changing the `files` constraint and dropping the auth annotation each redden the suite, and none of them did before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
The deployment endpoints are the public API surface, and the generated clients carry this title into their own documentation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
The spec describes one endpoint today, and five of these tests read it by unpacking a single item or by indexing get and post directly. The first endpoint added turns all five red for no reason, and a GET-only one raises KeyError. They now walk whatever the spec documents. The drift and diagnostics gates are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
chandrasekharan-zipstack
marked this pull request as ready for review
August 17, 2026 16:06
Contributor
|
| Filename | Overview |
|---|---|
| backend/api_v2/openapi_schema.py | Defines client-facing request, response, authentication, operation, and error metadata for the deployment execute and status endpoints. |
| backend/api_v2/management/commands/generate_docstudio_spec.py | Generates a deterministic committed schema and provides a non-writing drift-check mode. |
| backend/api_v2/deployment_spec_urls.py | Selects the deployment route from the served root URL configuration so generated paths retain their real mount prefix. |
| backend/api_v2/tests/test_docstudio_spec.py | Verifies committed-schema drift, route reachability, authentication metadata, operation coverage, failure responses, and response-field compatibility. |
| backend/api_v2/serializers.py | Adds schema metadata that represents multipart uploads as binary without changing request validation behavior. |
| backend/backend/settings/base.py | Configures drf-spectacular as DRF's schema generator and removes the legacy documentation app configuration. |
| backend/pyproject.toml | Replaces drf-yasg with a pinned drf-spectacular dependency to keep committed schema rendering deterministic. |
| specs/docstudio-oss.json | Commits the generated OpenAPI contract consumed by downstream deployment clients. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Runtime[Deployment views and serializers] --> Generator[drf-spectacular SchemaGenerator]
Routes[Deployment spec URLconf] --> Generator
Annotations[OpenAPI annotations] --> Generator
Generator --> Spec[specs/docstudio-oss.json]
Generator --> DriftTest[Schema drift tests]
Spec --> DriftTest
Spec --> Clients[Published client and generated SDK]
Reviews (2): Last reviewed commit: "test: name the downstream repos in the d..." | Re-trigger Greptile
chandrasekharan-zipstack
changed the title
The `docs` app served a redoc UI over a schema drf-yasg introspected with `public=False`, so an anonymous caller saw nothing and no endpoint carried a `@swagger_auto_schema` annotation. Nothing generates or consumes it. Removes the dependency, the `docs` app and its two mounts. The `/doc/` route goes with it; drf-spectacular serves no UI, only the committed spec. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
The annotation serializers exist only to shape the published spec, so they sit in `api_v2/openapi_schema.py` rather than in `serializers.py`, where a request-time import of one would look ordinary. `api_deployment_views.py` keeps a single decorator. `deployment_spec_urls.py` now selects mounts from a tuple, so widening the spec to another endpoint is one entry plus its `@extend_schema`. The generated spec is unchanged: component names are all that reach it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
…ponse fields The drift test is the only gate, so its message has to reach the person or agent who then has to regenerate the client and the CLI; it now names both repos, as does the management command's `--check`. Adds one binding the annotation could not express by inheritance: the view returns the execution DTO as a dict rather than through `APIExecutionResponseSerializer`, so a renamed DTO field would reach clients as a field the server never sends. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
|
Contributor
Unstract test resultsPer-group results
Critical paths
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The OpenAPI description of the API deployment
execute/statusendpoints is now generated and committed in this repository, and a test fails when it drifts from the code.drf-yasg, which generated nothing anyone read, is removed.Why
The published Python client and the SDK generated for it are built from that description. It was produced by a script living outside this repository, so a route, serializer or response change here could silently invalidate it — the breakage would surface later, in a client repo, against a spec nobody in the PR could see.
How
drf-spectacularadded as a backend dependency;DEFAULT_SCHEMA_CLASSandSPECTACULAR_SETTINGSset insettings/base.py. Both are read only during schema generation — no request-time behaviour changes.api_v2/openapi_schema.pyholds the schema annotation. The introspected schema is wrong in ways that matter to a generated client: a bareFileFieldmaps toformat: uri(right for output, wrong for a multipart upload),result: nullwhile an execution is pending crashes a generated deserialiser withoutallow_null, andoperation_id/tagsdecide the command names and module paths clients expose. It is a module of its own rather than part ofserializers.py, because none of these serializers parses a request or builds a response.api_v2/deployment_spec_urls.py— a urlconf mirroring the real mount. Generating against the included sub-urlconf drops the prefix and produces paths the server does not serve.SPEC_URLCONFSis a tuple: widening the spec to another endpoint is one entry plus its@extend_schema.manage.py generate_docstudio_specwritesspecs/docstudio-oss.jsonwith sorted keys, so the committed file is a usable drift signal.--checkfails instead of writing.api_v2/tests/test_docstudio_spec.pyregenerates and compares, and checks every documented operation resolves to a served URL, declares the deployment key, and documents the failures a client will branch on. It runs in the existingunit-backendgroup — no database, no new CI job. Its failure message names the repos that regenerate from the spec, since propagating the change downstream is the part CI cannot do.drf-yasgand thedocsapp removed. The redoc UI they served was built withpublic=Falseand no endpoint carried a@swagger_auto_schema, so an anonymous caller saw an empty schema. Nothing generated or consumed it.Regenerating
from
backend/, then commit. The test says so in its failure message, along with the client and CLI repos that regenerate from the result.Can this PR break any existing features
The
/doc/route is gone withdrf-yasg, and drf-spectacular serves no UI in its place — the spec is a committed file, not an endpoint. Nothing else changes at request time: the two new settings are read only during schema generation, and no route, serializer or response is modified. The annotation is metadata on the view, not behaviour.Database Migrations
None.
Env Config
None.
Notes on Testing
666 passed, 1 skippedin the backend unit tier this lands in, including the new spec tests. The generated spec is byte-identical to what the external script produced, apart from the roottagsarray — clients had nowhere to read group descriptions from, which is the one deliberate addition.Related Issues or PRs
🤖 Generated with Claude Code
https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ