Cost Guard Middleware

[veeqtoh]

Is your middleware request related to a problem? Please describe.

Production AI-native applications need to prevent runaway costs from unexpected token usage, model switches, or abuse. There is currently no way to enforce per-request, per-session, or per-user cost budgets within the Intercept middleware stack.

Proposed solution

A CostGuard middleware that estimates or tracks cost and enforces budgets before or after the provider call.

API Design

public function middleware(): array
{
    return [
        new CostGuard(
            maxCostPerRequest: 0.50, // USD
            maxCostPerSession: 10.00,
            maxCostPerUserPerDay: 5.00,
            modelPricing: null, // uses built-in pricing table or custom
            action: 'block', // 'block' | 'log' | 'truncate'
        ),
    ];
}

Behavior

• Uses RateLimiter::tooManyAttempts() under the hood.
• On exceed: throws RateLimitExceededException (extends InterceptException) or executes a custom callback.
• Respects Laravel's Retry-After headers if exposed to the client.

Consider shipping with a pricing file mapping common models (gpt-4o, claude-3-5-sonnet, etc.) to per-1K token rates. Allow users to override or extend.

Describe the middleware's supported actions you'd like

• block
• log
• truncate