Skip to content

refactor: Reduce copying of HTTP headers - #269

Merged
ayushag-nv merged 4 commits into
mainfrom
gk-headermap
Aug 4, 2026
Merged

refactor: Reduce copying of HTTP headers#269
ayushag-nv merged 4 commits into
mainfrom
gk-headermap

Conversation

@grahamking

@grahamking grahamking commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Use http::HeaderMap instead of BTreeMap. That means we can use what
axum gives switchyard-server directly.

Another benefit is http package already normalizes (to lower case) the
headers so we don't need that step.

In future we will also be able to handle headers with multiple values
(HeaderMap::get_all vs get).

Signed-off-by: Graham King grahamk@nvidia.com

Summary by CodeRabbit

  • Improvements
    • Improved HTTP header handling across request metadata, routing logs, Python bindings, and language-model requests.
    • Preserved original header names and values during forwarding and metadata processing.
    • Added validation for headers supplied through Python APIs, with clear errors for invalid values.
    • Maintained reserved-header filtering and authentication behavior.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-269/

Built to branch gh-pages at 2026-08-04 15:48 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@grahamking

Copy link
Copy Markdown
Contributor Author

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The PR adds the shared http dependency and replaces string-keyed header maps with typed http::HeaderMap values across metadata, server routing logs, LLM client forwarding, library tests, and Python bindings.

Changes

Typed header propagation

Layer / File(s) Summary
Metadata contract and header resolution
Cargo.toml, crates/protocol/Cargo.toml, crates/protocol/src/metadata.rs
Metadata stores and accepts http::HeaderMap. Header lookup, JSON-path resolution, and metadata tests use typed headers.
Server metadata and routing-log flow
crates/switchyard-server/Cargo.toml, crates/switchyard-server/src/lib.rs, crates/switchyard-server/src/routing_log.rs
The server passes original headers to metadata and routing-log construction. Invalid and empty routing headers are ignored.
LLM client header forwarding
crates/libsy-llm-client/Cargo.toml, crates/libsy-llm-client/src/client.rs
call_rewrite_model_raw accepts HeaderMap. Reserved-header filtering and forwarding tests use typed header values.
Library and Python API callers
crates/libsy/Cargo.toml, crates/libsy/src/algorithms/*, crates/switchyard-py/Cargo.toml, crates/switchyard-py/src/interop/subagent.rs, crates/switchyard-py/src/libsy_bindings.rs
Library tests and Python bindings convert string collections to HeaderMap. Python conversion errors return PyValueError where supported.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with headers, typed and neat,
HeaderMap makes each route complete.
Old string maps hop away,
Metadata guides the way.
Forwarded fields now safely meet.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: reducing HTTP header copying by using typed HeaderMap values.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
crates/protocol/src/metadata.rs (1)

196-198: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document Metadata::from_headers.

Metadata::from_headers is a public API. Replace // Create Metadata with a Rust documentation comment. State that the method parses recognized headers and does not retain the input map for forwarding.

As per coding guidelines, “Add concise Rust documentation comments for public structs, enums, methods, and module intent.”

Proposed change
 impl Metadata {
-    // Create Metadata
+    /// Builds metadata from recognized HTTP headers.
+    ///
+    /// The caller can retain `headers` in [`Self::http_headers`] when forwarding is required.
     pub fn from_headers(headers: &http::HeaderMap) -> Self {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/protocol/src/metadata.rs` around lines 196 - 198, Replace the `//
Create Metadata` comment above `Metadata::from_headers` with a concise Rust
documentation comment explaining that the method parses recognized headers and
does not retain the input `HeaderMap` for forwarding.

Source: Coding guidelines

crates/libsy-llm-client/src/client.rs (1)

1624-1633: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Cover repeated header values.

This fixture only uses HeaderMap::insert. Add a non-reserved header with two append calls, then assert that both values reach the upstream request. The documented HeaderMap iterator yields one item for each associated value, and RequestBuilder::header appends each item. (docs.rs)

Suggested regression coverage
         headers.insert(
             "accept-encoding",
             http::HeaderValue::from_static("gzip, br"),
         );
+        headers.append(
+            "x-test-multi",
+            http::HeaderValue::from_static("one"),
+        );
+        headers.append(
+            "x-test-multi",
+            http::HeaderValue::from_static("two"),
+        );
...
         assert!(!received.headers.contains_key("accept-encoding"));
+        assert_eq!(
+            received.headers.get_all("x-test-multi").iter().count(),
+            2
+        );
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy-llm-client/src/client.rs` around lines 1624 - 1633, Extend the
header fixture around the existing HeaderMap setup with a non-reserved header
added via two append calls, then assert the upstream request receives both
values. Verify the forwarding path iterates all associated values from the
HeaderMap and passes each through RequestBuilder::header rather than retaining
only one value.
crates/libsy/Cargo.toml (1)

19-19: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Move http.workspace = true to [dev-dependencies].

All http references are inside #[cfg(test)] modules. No production source uses http.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy/Cargo.toml` at line 19, Move the http dependency declaration
from the production dependency section into [dev-dependencies] in Cargo.toml,
preserving the workspace-based version setting so test-only references continue
to resolve.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/switchyard-py/src/interop/subagent.rs`:
- Around line 22-25: The shared Python-header conversion used by
is_subagent_request and PyAlgorithm::run must become fallible and bounded:
return PyResult<bool>, reject invalid names or values, case-insensitive
duplicate names, and capacity overflow, and map every failure to PyValueError
using fallible insertion rather than HeaderMap::extend/append. Update the
PyAlgorithm::run caller to propagate the result and add tests covering each
rejected input.

---

Nitpick comments:
In `@crates/libsy-llm-client/src/client.rs`:
- Around line 1624-1633: Extend the header fixture around the existing HeaderMap
setup with a non-reserved header added via two append calls, then assert the
upstream request receives both values. Verify the forwarding path iterates all
associated values from the HeaderMap and passes each through
RequestBuilder::header rather than retaining only one value.

In `@crates/libsy/Cargo.toml`:
- Line 19: Move the http dependency declaration from the production dependency
section into [dev-dependencies] in Cargo.toml, preserving the workspace-based
version setting so test-only references continue to resolve.

In `@crates/protocol/src/metadata.rs`:
- Around line 196-198: Replace the `// Create Metadata` comment above
`Metadata::from_headers` with a concise Rust documentation comment explaining
that the method parses recognized headers and does not retain the input
`HeaderMap` for forwarding.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: eb46242d-b4ad-4874-b15f-632721754934

📥 Commits

Reviewing files that changed from the base of the PR and between ad8635f and a649a3f.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock, !Cargo.lock
📒 Files selected for processing (14)
  • Cargo.toml
  • crates/libsy-llm-client/Cargo.toml
  • crates/libsy-llm-client/src/client.rs
  • crates/libsy/Cargo.toml
  • crates/libsy/src/algorithms/subagent_affinity_tests.rs
  • crates/libsy/src/algorithms/util/subagent.rs
  • crates/protocol/Cargo.toml
  • crates/protocol/src/metadata.rs
  • crates/switchyard-py/Cargo.toml
  • crates/switchyard-py/src/interop/subagent.rs
  • crates/switchyard-py/src/libsy_bindings.rs
  • crates/switchyard-server/Cargo.toml
  • crates/switchyard-server/src/lib.rs
  • crates/switchyard-server/src/routing_log.rs

Comment thread crates/switchyard-py/src/interop/subagent.rs Outdated
Use `http::HeaderMap` instead of `BTreeMap`. That means we can use what
`axum` gives `switchyard-server` directly.

Another benefit is `http` package already normalizes (to lower case) the
headers so we don't need that step.

In future we will also be able to handle headers with multiple values
(`HeaderMap::get_all` vs `get`).

Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: Graham King <grahamk@nvidia.com>
@grahamking
grahamking marked this pull request as ready for review August 4, 2026 15:42
@grahamking
grahamking requested a review from a team as a code owner August 4, 2026 15:42
Signed-off-by: Graham King <grahamk@nvidia.com>

@ayushag-nv ayushag-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@ayushag-nv
ayushag-nv merged commit 5ad7c0e into main Aug 4, 2026
20 checks passed
@ayushag-nv
ayushag-nv deleted the gk-headermap branch August 4, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants