UN-4010 [FEAT] Support every API deployment request parameter via a generated transport

[chandrasekharan-zipstack]

What

APIDeploymentsClient issues its requests through a transport generated from the API's OpenAPI spec instead of hand-built requests calls, and gains the ten request parameters the deployment accepts that had no argument to travel through.

Why

Every parameter the deployment accepts had to be added to this client by hand, so it lagged the API: of the twelve execute accepts, the client could send two, and only through the constructor. Generating the transport from the spec the backend now commits (Zipstack/unstract#2237) makes the wire format follow the API rather than a hand-maintained copy of it.

How

• specs/docstudio-oss.json + tools/gen_sdk.sh regenerate src/unstract/api_deployments/sdk_docstudio/ with a pinned generator. The generated tree is committed, marked linguist-generated, stamped DO-NOT-EDIT and excluded from lint — regeneration overwrites it wholesale, so fixes belong in client.py or in the spec.
• Only the innermost transport call was swapped. The retry policy, its wait strategy and every return shape are unchanged.
• httpx transport failures are translated to their requests equivalents inside the retried callable, so callers catching ConnectionError / Timeout still work and transport-error retry still counts.
• Requests carry only the fields this client sets. The generated builders write every spec default, and sending a default pins a value the server would otherwise choose.
• The status endpoint arrives in the execute reply, and only its path is taken. Joining an absolute one against the base URL would have let the reply decide which host receives the deployment key.
• The key is read per request rather than captured when the transport is built, so assigning api_key takes effect on the next call as it did when every call built its own header.
• The new parameters (c291e36 for structure_file, ed89066 for check_execution_status) are keyword-only, named exactly as the API names them, and unset by default — an unset parameter is not sent, so the request is byte-for-byte unchanged for every existing call shape. timeout and include_metadata still fall back to the constructor values; a per-request timeout selects the execution mode for that request. execution_id stays out: it is read from the endpoint URL the server handed back.

Can this PR break any existing features

Two deliberate behaviour changes, both narrow:

• File handles opened by structure_file are now closed after the request. The previous client leaked them.
• The status URL is rebuilt from the spec route plus the execution id rather than concatenating the server-supplied path. test_status_url_matches_the_released_client pins that the resulting request is identical.

Everything else is asserted equal to 1.5.3. New parameters are keyword-only and unset by default, so no released call shape changes.

Notes on Testing

372 tests. tests/test_compat.py compares this client against 1.5.3, vendored at tests/baseline/client_1_5_3.py and refreshed via tools/refresh_baseline.sh:

• constructor parameters, defaults, order, public method signatures and class attributes, read out of the released source by AST
• what goes out on the wire: send-only guards, multipart values, URLs, and that api_timeout (a backend execution mode, not a socket timeout) never reaches the transport
• httpx → requests exception translation, asserted on the exact class rather than with a bare pytest.raises — a bare one accepts a superclass, and it hid two real mismatches
• the exact dict each method returns, by running both clients over the same responses across 14 status/body cases
• that falsy values (0, False, "") are sent rather than filtered out as absent
• that a status endpoint naming another host is polled on the configured origin, and that a rotated api_key reaches the next request

The existing retry suite is unchanged apart from its patch target. A live round trip against a deployment is still outstanding.

Related Issues or PRs

• UN-4010 (sub-task of UN-4008)
• UN-4009 [MISC] Generate and commit the API deployment OpenAPI spec in-repo unstract#2237 — the spec this transport is generated from
• UN-4008 [FEAT] Add the unstract CLI to run extractions and API deployments from the terminal unstract-cli#2 — the CLI this unblocks

Dependencies Versions

Adds httpx. requests stays, as the exception types callers catch. Actions in test.yml are pinned to commit SHAs, matching clone-tests.yml.

🤖 Generated with Claude Code

https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ

[chandrasekharan-zipstack]

Backported a fix found by the sibling client's live run: httpx.ReadTimeout was falling into the TimeoutException catch-all and surfacing as requests.Timeout, where the published client raises requests.ReadTimeout. A caller catching ReadTimeout by name would have stopped matching. The translation-table test passed either way because pytest.raises is subclass-tolerant; it now asserts the exact class and fails on the previous code. 310 tests pass.

The live round trip for this client is still outstanding — it needs staging credentials.

[greptile-apps]

Greptile Summary

The PR replaces hand-built API deployment requests with a generated httpx transport while preserving the public response and retry behavior. It also exposes the deployment API’s additional request parameters and hardens status polling and CI dependency integrity.

• Generates and commits an OpenAPI-based deployment transport with a drift check.
• Adds keyword-only execution and status parameters while omitting unset fields.
• Translates httpx transport exceptions into the requests exception classes callers already handle.
• Constrains status polling to the configured origin and pins workflow actions to immutable commits.
• Removes the conflicting unstract console-script registration and documents module-based clone invocation.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/unstract/api_deployments/client.py	Routes deployment calls through the generated transport, forwards newly supported parameters, preserves retry and response compatibility, and reconstructs polling URLs on the configured origin.
specs/docstudio-oss.json	Defines the deployment execute and status contracts used to generate the committed transport.
src/unstract/api_deployments/sdk_docstudio/client.py	Provides the generated authenticated httpx client used by the compatibility wrapper.
tests/test_compat.py	Exercises public API and wire compatibility, parameter forwarding, exception translation, credential rotation, and same-origin status polling.
.github/workflows/test.yml	Pins third-party actions to immutable commits and adds generated-SDK drift detection.
pyproject.toml	Adds bounded generated-transport dependencies, excludes generated code from linting, and removes the conflicting console-script entry.
uv.lock	Locks the newly introduced transport dependencies and updated project dependency constraints.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Client as APIDeploymentsClient
    participant SDK as Generated transport
    participant API as Configured deployment origin
    Caller->>Client: structure_file(files, options)
    Client->>SDK: Build multipart request
    Client->>API: Execute with bearer token
    API-->>Client: status_api with execution_id
    Caller->>Client: check_execution_status(status_api)
    Client->>Client: Extract execution_id and discard supplied origin
    Client->>API: Poll reconstructed same-origin URL
    API-->>Caller: Normalized result dictionary

Loading

_{Reviews (3): Last reviewed commit: "UN-4010 [MISC] pin the origin against ev..." | Re-trigger Greptile}

[greptile-apps]

Scheme-relative path leaks credentials

When a returned status endpoint has a path beginning with //, urljoin interprets the stripped path as a replacement host, causing _send to transmit the deployment bearer token to that host.

How this was verified: The status endpoint flows through _status_url, its preserved // path reaches urljoin, and _send attaches the bearer token to the resulting host.

Suggested change

	return urljoin(
	self.base_url,
	urlparse(endpoint)._replace(scheme="", netloc="", query="").geturl(),
	)
	safe_path = "/" + urlparse(endpoint).path.lstrip("/")
	return urljoin(self.base_url, safe_path)

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/unstract/api_deployments/client.py
Line: 336-339

Comment:
**Scheme-relative path leaks credentials**

When a returned status endpoint has a path beginning with `//`, `urljoin` interprets the stripped path as a replacement host, causing `_send` to transmit the deployment bearer token to that host.

**How this was verified:** The status endpoint flows through `_status_url`, its preserved `//` path reaches `urljoin`, and `_send` attaches the bearer token to the resulting host.

```suggestion
        safe_path = "/" + urlparse(endpoint).path.lstrip("/")
        return urljoin(self.base_url, safe_path)
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[chandrasekharan-zipstack]

This one does not reproduce. _replace(netloc="") is applied before geturl(), and urlunparse re-prefixes a path that begins // with another // precisely so it cannot be re-read as an authority — so urljoin resolves it as a path.

Checked end to end through check_execution_status, deployment URL https://api.example.com/other/testorg/testapi/ (the branch that reads the endpoint rather than rebuilding it):

returned endpoint	request actually issued
https://attacker.example/x	https://api.example.com/x
//attacker.example/x	https://api.example.com/x
///attacker.example/x	https://api.example.com/attacker.example/x
////attacker.example/x	https://api.example.com//attacker.example/x
https:////attacker.example/x	https://api.example.com//attacker.example/x
https://api.example.com//attacker.example/x	https://api.example.com//attacker.example/x

Every one stays on the configured origin, so the token does not travel.

The suggested "/" + urlparse(endpoint).path.lstrip("/") is not safer here — it maps //attacker.example/x to /attacker.example/x, where the current code gives /x.

Rather than change the code I have pinned the property: d67dda4 parametrises test_a_status_endpoint_naming_another_host_is_not_polled over all six spellings above. Reverting the origin restriction fails it on the two spellings that were genuinely reachable, which is the report you filed first and which is fixed.

[greptile-apps]

You're right — I traced this incorrectly. urlparse("//attacker.example/x") parses attacker.example as the netloc and /x as the path, so the _replace(netloc="") step discards the host before geturl() runs, leaving only /x for urljoin to resolve. The scheme-stripping in the same _replace call neutralises the https://attacker.example/x case at the same time. The code handles all the spellings your table covers.

The parametrised test on d67dda4 is the right way to lock this in — it fails on a revert of the origin restriction and covers all six variants. Happy to withdraw the comment.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.