diff --git a/docs/about-nemo-relay/agent-runtime-primer.mdx b/docs/about-nemo-relay/agent-runtime-primer.mdx index 2209a16fa..5567545fb 100644 --- a/docs/about-nemo-relay/agent-runtime-primer.mdx +++ b/docs/about-nemo-relay/agent-runtime-primer.mdx @@ -1,7 +1,7 @@ --- title: "Agent Runtime Primer" description: "Learn the NeMo Relay runtime model for scopes, middleware, events, and integration boundaries." -position: 2 +position: 3 --- import { MermaidStyles } from "@/components/MermaidStyles"; diff --git a/docs/about-nemo-relay/architecture.mdx b/docs/about-nemo-relay/architecture.mdx index 8af4a69d2..937ed5d77 100644 --- a/docs/about-nemo-relay/architecture.mdx +++ b/docs/about-nemo-relay/architecture.mdx @@ -1,7 +1,7 @@ --- title: "Architecture" description: "" -position: 3 +position: 4 --- import { MermaidStyles } from "@/components/MermaidStyles"; diff --git a/docs/about-nemo-relay/concepts/codecs.mdx b/docs/about-nemo-relay/concepts/codecs.mdx index 0049d840e..39f64c23b 100644 --- a/docs/about-nemo-relay/concepts/codecs.mdx +++ b/docs/about-nemo-relay/concepts/codecs.mdx @@ -1,7 +1,7 @@ --- title: "Codecs" description: "" -position: 7 +position: 6 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} diff --git a/docs/about-nemo-relay/concepts/events.mdx b/docs/about-nemo-relay/concepts/events.mdx index 5c6221155..b696766ed 100644 --- a/docs/about-nemo-relay/concepts/events.mdx +++ b/docs/about-nemo-relay/concepts/events.mdx @@ -1,7 +1,7 @@ --- title: "Events" description: "" -position: 4 +position: 2 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} diff --git a/docs/about-nemo-relay/concepts/framework-integrations.mdx b/docs/about-nemo-relay/concepts/framework-integrations.mdx deleted file mode 100644 index c948f421b..000000000 --- a/docs/about-nemo-relay/concepts/framework-integrations.mdx +++ /dev/null @@ -1,168 +0,0 @@ ---- -title: "Framework Integrations" -description: "" -position: 6 ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - -This page explains how framework integrations should attach existing application work to -NeMo Relay runtime semantics. - -## Why Framework Integrations Are Different - -Application code can usually call the managed NeMo Relay helpers directly. -Framework integrations often cannot. - -A framework may already own: - -- The real invocation boundary -- The scheduling model -- The retry loop -- The callback signature -- The provider payload shape - -That means framework integrations must choose the best instrumentation boundary -available rather than assuming direct runtime ownership. - -## Preferred Integration Order - -When integrating NeMo Relay into an existing framework, prefer these choices in -order: - -1. Execution wrappers through managed execute helpers -2. Explicit API calls for lifecycle emission, conditional execution, or request intercepts -3. Mark events only - -This order preserves the most runtime semantics with the least distortion. - -This order also keeps ownership clear. A framework integration should preserve -the framework's scheduling, retry, provider routing, and object-lifetime rules -unless a managed NeMo Relay wrapper explicitly owns that invocation boundary. - -## First Choice: Execution Wrappers - -Execution wrappers are the preferred integration boundary when a framework exposes a -real callback or handler. - -### Managed Execute Helpers - -Use the managed execute helpers when the framework exposes a stable callable -boundary that NeMo Relay can wrap. - -### Why This Is Preferred - -This is the best integration shape because it preserves: - -- Correct lifecycle ordering -- The full middleware pipeline -- Natural parent-child scope relationships -- The cleanest wrapper point for retries, routing, and timing - -Execution wrappers are also the natural place to align framework semantics with -NeMo Relay execution intercepts. - -## Fallback: Explicit API Calls - -Use explicit API calls when the framework owns part of the invocation lifecycle -and cannot hand NeMo Relay a stable callback to wrap. Explicit calls let the -framework keep its own scheduler, retry loop, callback signature, or provider -client while still using selected NeMo Relay runtime behavior. - -### What You Lose From Managed Execution Wrappers - -Explicit API calls are useful, but they are narrower than managed execution -wrappers. Depending on which explicit APIs you call, you can lose: - -- Automatic start-to-end lifecycle pairing -- Automatic execution-intercept chaining around the real callback -- Automatic request and response guardrail placement -- One canonical parent-child relationship for the wrapped span -- One call site that applies the full middleware pipeline - -Use explicit APIs when they match the framework boundary. Prefer managed -execution wrappers whenever the framework can expose the real callback. - -### Explicit Start, End, and Mark Emission - -Use explicit start and end emission when the framework gives reliable lifecycle -hooks but does not let NeMo Relay wrap the real invocation. - -1. Call the explicit start API as early as the framework can identify the work. -2. Retain the returned handle. -3. Call the matching end API when the work succeeds or fails. -4. Emit mark events for milestones that are important but are not full tool or - LLM calls. - -This fallback preserves lifecycle visibility, but the framework must pair start -and end calls correctly. - -Manual lifecycle calls do not run the full managed execution pipeline by -themselves. They preserve observability and parentage, but execution intercepts, -request intercepts, and sanitize guardrails only run when the integration calls -the corresponding managed or standalone runtime surface. - -### Conditional Execution - -Use standalone conditional-execution helpers when the framework only needs an -allow-or-block decision before continuing its own invocation path. - -This is the preferred explicit API when the framework can ask NeMo Relay for a -policy decision but must still execute the real tool or provider call itself. -The helper returns the guardrail decision; it does not emit a full managed -lifecycle span by itself. - -### Request Intercepts - -Use standalone request-intercept helpers when the framework needs NeMo Relay to -rewrite the request before the framework continues execution on its own. - -This is the preferred explicit API when the framework owns execution but can -accept a rewritten JSON-compatible request before it calls the underlying tool -or provider. Request-intercept helpers apply request transformation without -owning callback execution. - -Use mark events when the framework exposes important milestones but not a clean -start/end lifecycle boundary. - -Mark events are useful for: - -- Retries -- Queue transitions -- Scheduler milestones -- State changes -- Debugging checkpoints - -They provide visibility, but they are not a replacement for full lifecycle -instrumentation. - -## Choosing the Right Integration Boundary - -Use these rules to decide where NeMo Relay should wrap framework behavior. - -- If you can wrap the real callback, use managed execute helpers. -- If you cannot wrap the callback but you do have reliable start and end hooks, - use explicit lifecycle APIs. -- If you only need a block/allow decision, use conditional-execution helpers. -- If you only need request transformation, use request-intercept helpers. -- If you only have milestone visibility, emit mark events. - -If the LLM provider request or response payloads matter, we recommend using NeMo -Relay codecs and annotated request or response data before introducing ad hoc -raw-payload parsing in the integration. Ensure provider-specific round-trip -behavior stays in the codec or adapter that owns that provider shape. - -## Practical Guidance - -Use these practices when applying the concept in application or integration code. - -- Prefer execution wrappers over explicit helper calls whenever the framework - allows it. -- Treat explicit lifecycle calls as the main fallback for framework-owned invocation. -- Use conditional-execution functions and request-intercept helpers before - continuing framework-owned execution when you need policy or transformation - without managed callback wrapping. -- Use mark events to fill visibility gaps rather than to model full execution - spans. -- Keep binding-level API details in the [API Reference](/reference/api) and - deeper integration patterns in [Integrate into Frameworks](/integrate-into-frameworks/about). diff --git a/docs/about-nemo-relay/concepts/index.mdx b/docs/about-nemo-relay/concepts/index.mdx index 0ba4db236..682e4a619 100644 --- a/docs/about-nemo-relay/concepts/index.mdx +++ b/docs/about-nemo-relay/concepts/index.mdx @@ -13,10 +13,10 @@ in a use-case workflow. NeMo Relay's shared runtime model has five parts: - **Scopes** decide where work belongs and which scope-local behavior is visible. +- **Events** record what happened in the canonical lifecycle stream. - **Middleware** decides what can block, rewrite, sanitize, or wrap managed execution. - **Plugins** install reusable runtime behavior from configuration. -- **Events** record what happened in the canonical lifecycle stream. - **Subscribers and exporters** consume that stream in process, write raw ATOF events, or project events into downstream formats. @@ -38,7 +38,17 @@ runtime concept or integration boundary you need. href="/about-nemo-relay/concepts/scopes" > -Ownership boundaries for agent runs, requests, workflows, tools, LLM calls, and nested runtime work. +Scope hierarchy, lifetime, and isolation for agent runs, requests, tools, LLM calls, and nested work. + + + + +Canonical lifecycle records for scopes, tool calls, LLM calls, marks, subscribers, and exporters. - - -Canonical lifecycle records for scopes, tool calls, LLM calls, marks, subscribers, and exporters. - - - -Integration patterns for frameworks that own invocation boundaries, scheduling, retries, or provider payloads. - - - -Normalization boundaries for typed application values, provider payloads, middleware, and exporters. +Provider request and response normalization for middleware and observability. diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 19d583755..22d86914d 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -1,7 +1,7 @@ --- title: "Middleware" description: "" -position: 2 +position: 3 --- import { MermaidStyles } from "@/components/MermaidStyles"; diff --git a/docs/about-nemo-relay/concepts/plugins.mdx b/docs/about-nemo-relay/concepts/plugins.mdx index bb5f06528..7115e0ef1 100644 --- a/docs/about-nemo-relay/concepts/plugins.mdx +++ b/docs/about-nemo-relay/concepts/plugins.mdx @@ -1,7 +1,7 @@ --- title: "Plugins" description: "Understand NeMo Relay plugin configuration, lifecycle, ownership, and built-in components." -position: 3 +position: 4 --- import { MermaidStyles } from "@/components/MermaidStyles"; diff --git a/docs/about-nemo-relay/ecosystem.mdx b/docs/about-nemo-relay/ecosystem.mdx index afabe2329..a6b6cfab7 100644 --- a/docs/about-nemo-relay/ecosystem.mdx +++ b/docs/about-nemo-relay/ecosystem.mdx @@ -1,7 +1,7 @@ --- title: "Ecosystem" description: "Understand how NeMo Relay fits with agent frameworks, providers, and the NVIDIA NeMo ecosystem." -position: 4 +position: 2 --- import { MermaidStyles } from "@/components/MermaidStyles"; diff --git a/docs/contribute/runtime-contract-docs.mdx b/docs/contribute/runtime-contract-docs.mdx index 39679fffd..011d743c0 100644 --- a/docs/contribute/runtime-contract-docs.mdx +++ b/docs/contribute/runtime-contract-docs.mdx @@ -25,7 +25,6 @@ These pages define or route the shared runtime model: - `docs/about-nemo-relay/concepts/subscribers.mdx` - `docs/about-nemo-relay/concepts/plugins.mdx` - `docs/about-nemo-relay/concepts/events.mdx` -- `docs/about-nemo-relay/concepts/framework-integrations.mdx` - `docs/about-nemo-relay/concepts/codecs.mdx` - `docs/about-nemo-relay/agent-runtime-primer.mdx` - `docs/getting-started/about.mdx` diff --git a/docs/getting-started/quick-start/index.mdx b/docs/getting-started/quick-start/index.mdx index 972debb96..9597c4d86 100644 --- a/docs/getting-started/quick-start/index.mdx +++ b/docs/getting-started/quick-start/index.mdx @@ -95,7 +95,7 @@ agent lifecycle hooks. If you are building or reviewing a framework integration that is not covered by a maintained guide, use the generic framework integration workflow instead. - + Choose the generic workflow for unsupported frameworks, provider adapters, or -new public-API integrations. - - - - -Ensure you understand what the framework owns, what Relay owns, and which -runtime semantics each integration boundary preserves. +new public API integrations, including how to choose the Relay APIs that match +the work your framework controls. diff --git a/docs/integrate-into-frameworks/about.mdx b/docs/integrate-into-frameworks/about.mdx index 7b6a2ed00..d921474a5 100644 --- a/docs/integrate-into-frameworks/about.mdx +++ b/docs/integrate-into-frameworks/about.mdx @@ -1,6 +1,6 @@ --- title: "About" -description: "" +description: "Choose NeMo Relay APIs for tool and LLM calls controlled by a framework." position: 1 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. @@ -20,6 +20,63 @@ Prefer a managed execution wrapper around a stable tool or LLM callback. When that is not possible, use explicit lifecycle calls, standalone guardrail or intercept helpers, or mark events. +## Choose How Relay Connects + +Prefer these options in order when they represent the lifecycle boundary you +need. Move down the table when the option above cannot capture that boundary or +the framework does not support it. + +| Method | Use When | What Relay Handles | +| --- | --- | --- | +| Managed execution wrapper | Relay can wrap the real tool or LLM callback. | Lifecycle pairing, the complete middleware pipeline, timing, and event parentage. | +| Explicit start and end calls | The framework exposes reliable lifecycle hooks but owns callback execution. | Event construction and parentage; the integration must pair the calls. | +| Standalone conditional-execution helper | The framework needs an allow-or-block decision before its own call. | The policy decision only. | +| Standalone request-intercept helper | The framework can accept a rewritten request before its own call. | Request transformation only. | +| Mark event | The framework exposes only a milestone or state transition. | One point-in-time event without a duration. | + +### Prefer Managed Execution Wrappers + +A managed wrapper is the best fit when the framework exposes a stable callback. +It preserves lifecycle ordering, middleware placement, timing, and +parent-child relationships around the real invocation. The framework should +still own its scheduler, retries, routing, and object lifetimes unless the +wrapper explicitly encloses that behavior. + +### Use Explicit Lifecycle Calls as the Main Fallback + +When Relay cannot wrap the callback, use reliable framework hooks to emit the +lifecycle explicitly: + +1. Emit the start event as soon as the framework can identify the work. +2. Retain the returned handle. +3. Emit the matching end event for success or failure. +4. Use marks only for milestones that do not have a complete lifecycle. + +Manual lifecycle calls preserve observability and parentage. They do not run +execution intercepts, request intercepts, or request and response guardrails +unless the integration invokes those standalone APIs separately. + +### Use Standalone Policy or Transformation When Needed + +Conditional-execution helpers return an allow-or-block decision before the +framework continues its own invocation. Request-intercept helpers return a +rewritten JSON-compatible request before framework-owned execution. Neither +helper creates a complete managed lifecycle span by itself. + +### Use Marks for Checkpoints + +Marks fit retries, queue transitions, scheduler milestones, state changes, and +debugging checkpoints. They improve visibility but do not replace paired start +and end events. + +### Keep Provider Semantics in Codecs + +When provider request or response payloads matter, use +[Provider Codecs](/integrate-into-frameworks/provider-codecs) and annotated +request or response data before adding raw-payload parsing to the integration. +Provider-specific round-trip behavior should remain in the codec or adapter for +that provider. + ## Start Here Use these signals to decide whether this documentation path matches your current task. diff --git a/docs/resources/support-and-faqs.mdx b/docs/resources/support-and-faqs.mdx index 37cf44214..b1b0e0ed9 100644 --- a/docs/resources/support-and-faqs.mdx +++ b/docs/resources/support-and-faqs.mdx @@ -402,8 +402,7 @@ owns execution. Use standalone conditional-execution or request-intercept helpers only when the framework needs those decisions before it invokes its own downstream code. -Refer to [Framework Integrations](/about-nemo-relay/concepts/framework-integrations) and -[Integrate into Frameworks](/integrate-into-frameworks/about). +Refer to [Integrate into Frameworks](/integrate-into-frameworks/about). ### How Does NeMo Relay Connect To My Favorite Agent Harness Or Framework? diff --git a/fern/docs.yml b/fern/docs.yml index 2f46f4823..feb89ebe5 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -18,6 +18,10 @@ announcement: message: "🔔 NVIDIA NeMo Relay is beta software. APIs and behavior may change with each release." redirects: +# Concept relocations +- source: /nemo/relay/about-nemo-relay/concepts/framework-integrations + destination: /nemo/relay/integrate-into-frameworks/about + # Release notes - source: /nemo/relay/about-nemo-relay/release-notes/related-topics destination: /nemo/relay/about-nemo-relay/release-notes