Goal
NeMo Guardrails should act as a trustworthy proxy: failures from generation
models, rail models, and rail APIs must be attributed correctly, represented
safely, and translated consistently for API clients.
Why
The current error paths do not always retain enough structure or provenance to
distinguish who caused a failure. For example:
- A self-check rail model returning 429 can appear as if the API caller
exhausted their own quota.
- A model rejecting the server's API key can return 401 as if the caller
supplied invalid credentials.
- IORails can lose provider metadata or expose internal upstream details.
- Serialized streaming error markers can be imitated by model output.
- Operational rail failures do not produce complete observability records.
These behaviors make errors difficult for clients to interpret safely and
correctly.
Scope
Sub-issues under this epic should address:
- failure provenance and outward status policy
- generation-model versus rail-model and rail-API failures
- server-owned versus caller-owned credentials and quotas
- safe, structured errors across LLMRails and IORails
- preservation of provider codes, parameters, and retry metadata
- typed streaming errors that model-controlled content cannot forge
- protocol-specific error rendering at the server boundary
- complete observability for failed rail calls
Completion criteria
- Client-attributable failures and server dependency failures are
distinguishable.
- Server-owned authentication or quota failures are not misrepresented as
caller failures.
- Client responses do not expose credentials, raw upstream bodies, internal
URLs, or routing details.
- Safe provider error metadata and retry information are preserved.
- Model output cannot create a terminal streaming control event.
- LLMRails and IORails follow the same approved error policy.
- Public API endpoints render errors according to their own protocol contract.
- Operational rail failures are represented correctly in spans and generation
logs.
Goal
NeMo Guardrails should act as a trustworthy proxy: failures from generation
models, rail models, and rail APIs must be attributed correctly, represented
safely, and translated consistently for API clients.
Why
The current error paths do not always retain enough structure or provenance to
distinguish who caused a failure. For example:
exhausted their own quota.
supplied invalid credentials.
These behaviors make errors difficult for clients to interpret safely and
correctly.
Scope
Sub-issues under this epic should address:
Completion criteria
distinguishable.
caller failures.
URLs, or routing details.
logs.