Skip to content

epic: make downstream failures safe and reliable across Guardrails APIs #2230

Description

@Pouyanpi

Goal

NeMo Guardrails should act as a trustworthy proxy: failures from generation
models, rail models, and rail APIs must be attributed correctly, represented
safely, and translated consistently for API clients.

Why

The current error paths do not always retain enough structure or provenance to
distinguish who caused a failure. For example:

  • A self-check rail model returning 429 can appear as if the API caller
    exhausted their own quota.
  • A model rejecting the server's API key can return 401 as if the caller
    supplied invalid credentials.
  • IORails can lose provider metadata or expose internal upstream details.
  • Serialized streaming error markers can be imitated by model output.
  • Operational rail failures do not produce complete observability records.

These behaviors make errors difficult for clients to interpret safely and
correctly.

Scope

Sub-issues under this epic should address:

  • failure provenance and outward status policy
  • generation-model versus rail-model and rail-API failures
  • server-owned versus caller-owned credentials and quotas
  • safe, structured errors across LLMRails and IORails
  • preservation of provider codes, parameters, and retry metadata
  • typed streaming errors that model-controlled content cannot forge
  • protocol-specific error rendering at the server boundary
  • complete observability for failed rail calls

Completion criteria

  • Client-attributable failures and server dependency failures are
    distinguishable.
  • Server-owned authentication or quota failures are not misrepresented as
    caller failures.
  • Client responses do not expose credentials, raw upstream bodies, internal
    URLs, or routing details.
  • Safe provider error metadata and retry information are preserved.
  • Model output cannot create a terminal streaming control event.
  • LLMRails and IORails follow the same approved error policy.
  • Public API endpoints render errors according to their own protocol contract.
  • Operational rail failures are represented correctly in spans and generation
    logs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    status: needs triageNew issues that have not yet been reviewed or categorized.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions