Skip to content

SharvikS/LLM-Firewall

BranchesTags

Repository files navigation

TITAN Gateway dashboard

TITAN Gateway

Zero-Trust LLM Security Gateway

Put one enforceable security layer between your apps, your users, and every LLM provider.

TITAN is an OpenAI-compatible reverse proxy and enterprise control plane for LLM traffic. It inspects prompts before they leave your boundary, masks PII/secrets, blocks prompt injection, enforces policy, controls cost, records audit evidence, and gives security teams one place to govern AI usage.

Go Python Next.js Kafka License

Why TITAN Exists

Enterprises are adopting LLMs faster than their security controls can follow. Applications send sensitive prompts to external models, employees paste data into web chat tools, and teams often have little evidence of what happened after the fact.

TITAN turns LLM usage into a governed security control:

Problem TITAN control
Prompt injection and jailbreaks ML + heuristic detection, custom guardrails, policy deny
PII, secrets, credentials, source-code leaks Request masking, output scanning, Browser DLP
Shadow AI web usage Browser extension for ChatGPT, Claude, Gemini, Perplexity
Unbounded model spend Tenant API keys, RPM/TPM limits, monthly quotas, billing views
Weak governance Cedar policy, RBAC, SSO, policy playground, version history
Missing audit evidence Kafka/DB audit trail, SIEM export, compliance reports

What You Get

Drop-in LLM firewall

  • OpenAI-compatible /v1/* proxy for OpenAI, Anthropic, Groq, Gemini, Ollama, LM Studio, vLLM, and other compatible providers.
  • Existing SDKs keep working. Change base_url and API key.
  • Provider failover replays the masked request body, not raw sensitive content.
  • Firewall API keys can carry sandbox restrictions, so one app key can be limited to approved models, API paths, per-key RPM/TPM caps, and required privacy controls.

Security enforcement

  • Prompt-injection and jailbreak detection.
  • Per-message PII masking with Microsoft Presidio.
  • Secret and source-code leak prevention.
  • Toxicity detection.
  • Response-side output scanning, including streaming responses.
  • Default-deny Cedar policy engine.
  • Dashboard-managed no-code guardrails.
  • API-key sandbox profiles for model/path/rate/privacy restrictions.
  • Sandboxed WASM custom detectors in Enterprise builds.
  • ASR-backed agent tool sandbox with Firecracker/Docker/simulated runtime selection, cancellation, and audit evidence.

Enterprise control plane

  • Next.js dashboard with 17 operational tabs.
  • Session auth, OIDC SSO, and four-tier RBAC.
  • Tenant-scoped API keys and per-tenant settings.
  • Usage metering, plan quotas, and billing visibility.
  • Policy playground and immutable policy version history.
  • Audit export, compliance coverage, and SIEM/webhook delivery.

Endpoint DLP

  • Cross-browser extension for AI web UIs.
  • Scans prompts, paste events, files, and images before upload.
  • Reports metadata-only events into the same audit, dashboard, and alerting surfaces as API traffic.

Architecture At A Glance

Apps / SDKs / Browser DLP
        |
        v
TITAN Go Gateway  ->  ML Engine  ->  Upstream LLMs
        |
        +--> ASR: isolated agent tool execution
        +--> Redis: rate limits, cache, usage
        +--> CockroachDB: tenants, keys, policies, audit, settings
        +--> Kafka/Redpanda + ClickHouse: audit stream and analytics
        +--> Qdrant: semantic cache
        +--> Dashboard: RBAC, settings, policies, audit, compliance

The gateway data plane is a single Go binary. The intelligence plane is a Python service for ML detection, PII masking, embeddings, file scanning, and Browser DLP side-channel endpoints. The control plane is a Next.js dashboard.

For the deeper design and threat model, read:

Fastest Path: Hosted TITAN

For most buyers, TITAN should work like ordinary SaaS:

  1. Choose Free, Starter, or Pro on the landing site.
  2. Paid plans complete Stripe Checkout.
  3. /start creates the workspace, dashboard user, and first app API key.
  4. Point an OpenAI-compatible SDK at the hosted gateway URL.

The storefront provisioning route is in landing/src/app/api/activate/route.ts. Set TITAN_CLOUD_GATEWAY_URL, TITAN_CLOUD_ADMIN_TOKEN, NEXT_PUBLIC_TITAN_DASHBOARD_URL, and NEXT_PUBLIC_TITAN_GATEWAY_BASE_URL on the landing deployment to enable it. See landing/README.md.

Self-Hosted Quick Start

Prerequisites: Docker 24+ and Docker Compose v2.

git clone https://github.com/SharvikS/LLM-Firewall.git
cd LLM-Firewall
./scripts/quickstart.sh

The quickstart creates missing env files, generates a shared admin token, optionally saves your provider API key, starts the full Docker stack, waits for the dashboard and gateway, and prints the next commands. Existing .env files are preserved.

Open the dashboard:

Surface URL
Dashboard http://localhost:3000
Gateway http://localhost:8080
API docs http://localhost:8080/docs
Grafana http://localhost:3001
Redpanda Console http://localhost:8082

Default local dashboard login:

admin@titan.local / admin@123

Seeded local API key:

titan_dev_localkeyfortesting1234

Run the smoke test after the stack is healthy:

./scripts/smoke.sh

Manual fallback:

cp .env.example .env
cp gateway/.env.example gateway/.env
docker compose up -d --build

For the complete demo flow, use docs/operations/DEMO_RUNBOOK.md.

Use It With Existing SDKs

TITAN is OpenAI-compatible. Point your client at the gateway:

from openai import OpenAI

client = OpenAI(
    api_key="titan_dev_localkeyfortesting1234",
    base_url="http://localhost:8080/v1",
)

All security controls run before the request reaches the upstream provider. For SDK examples, see sdk/python and sdk/node.

Dashboard

The dashboard is the day-to-day control plane for platform, security, and compliance teams.

Area What it manages
Overview, Events, Analytics Live traffic, risk, latency, model usage, blocked requests
Policy Engine Cedar policies, playground evaluation, version history
Browser DLP, DLP Flags Endpoint-side web UI blocks, redactions, repeat offenders
Audit Logs, Coverage Audit search/export and OWASP/NIST coverage evidence
Sandboxes ASR tool execution ledger, launch/cancel controls, backend/risk output
Settings, Edge Routing Upstreams, guardrails, alerting, rate limits, per-tenant overrides
Team, API Keys, Billing RBAC users, tenant keys, sandbox profiles, usage metering, plans and quotas

Product Tour

The screenshots below show the main buyer/operator workflows: executive overview, investigation, policy governance, access control, audit, and routing.

Command Center

TITAN overview dashboard
Overview — live KPIs, blocked traffic, cache rate, latency, and recent threats. 		TITAN analytics dashboard
Analytics — request volume, risk categories, latency, and model usage.

Investigation And Governance

TITAN events feed
Events — real-time blocks, masks, Browser DLP events, and risk reasons. 		TITAN policy engine
Policy Engine — create and evaluate Cedar policies without redeploying.
TITAN audit logs
Audit Logs — searchable evidence for requests and control-plane actions. 		TITAN API keys
API Keys — generate, copy, monitor, and revoke tenant-scoped keys.

Operations

TITAN edge routing
Edge Routing — switch upstream providers and test gateway reachability. 		TITAN settings
Settings — guardrails, alerting, security defaults, and tenant overrides.

TITAN analytics light theme
Light theme — same operational surface for customer demos and stakeholder reviews.

Security And Compliance Posture

TITAN is designed around these invariants:

  • Invalid or missing API keys never reach an upstream model.
  • API-key sandbox restrictions are enforced before provider routing.
  • Policy is default-deny; explicit denies win.
  • Detected PII/secrets are masked before provider calls and failover retries.
  • Raw prompts are not persisted in audit logs.
  • Control-plane mutations are RBAC-gated and audit-recorded.
  • Agent tool execution can be routed through ASR and recorded with backend, status, risk score, operator, and reason.
  • Kafka is the preferred audit path; DB fallback preserves audit events when Kafka is unavailable or produce fails.
  • Browser DLP reports metadata only, never prompt/file contents.
  • Production mode rejects public default secrets.

Current proof material:

Dependency and release gates run in GitHub Actions:

  • Go tests and Enterprise build-tag tests.
  • Dashboard lint, typecheck, and production build.
  • ML pytest.
  • Browser extension lint, tests, and package build.
  • Helm lint/template.
  • Dependency audits.
  • Load-test harness compile.

Browser DLP

The gateway protects API traffic. The Browser DLP extension protects human AI web usage where traffic does not pass through the gateway.

It supports:

  • ChatGPT, Claude, Gemini, and Perplexity.
  • Prompt and paste scanning.
  • File/image scanning with text extraction and OCR.
  • Block, auto-redact, and warn-only modes.
  • Fleet baseline controls from the dashboard.
  • Unified audit, event feed, DLP flags, and SOC alerting.

Full extension docs: browser-extension/README.md.

Operations

Need Start here
Local demo Demo Runbook
Buyer walkthrough Enterprise Demo Script
Kubernetes deployment Production Deployment Guide
Helm chart helm/titan
Backup and restore DR Runbook
Testing guide Enterprise Testing Guide
Providers Provider Reference
Security packet Buyer Security Packet

Repository Map

Path Purpose
gateway Go data plane, Admin API, auth, policy, audit, proxy
ml_engine Python analyzer, masking, embeddings, file/image scanning
dashboard Next.js control plane
browser-extension Cross-browser endpoint DLP extension
sdk/python, sdk/node Admin SDKs
helm/titan Kubernetes Helm chart
loadtest Load/stress test harness
docs Architecture, operations, security, testing

Current Readiness

TITAN is ready for demos, design-partner pilots, and controlled enterprise validation. The codebase includes the core security controls, dashboard, audit/compliance surfaces, deployment docs, and CI release gates needed to start serious buyer conversations.

Before claiming broad production SLOs, run and archive:

  • A production-like Helm deployment.
  • The load-test matrix in LOAD_SLO_REPORT.md.
  • Fresh dependency scan output from CI.
  • Customer-specific SSO, SIEM, backup, and restore evidence.
  • Third-party security review or pentest evidence.

Editions And License

TITAN is open-core.

  • Community: MIT-licensed gateway core, detection, masking, policy, dashboard basics, SDKs, cache, guardrails, and output scanning.
  • Enterprise: commercially licensed build-tagged features such as OIDC SSO, metering/quotas, SOC alerting, compliance export, WASM plugins, and groundedness scoring.
go build ./...                  # Community build
go build -tags enterprise ./... # Enterprise build, license-gated at runtime

See EDITIONS.md, LICENSE, and LICENSE-ENTERPRISE.md.

Contributing And Security

Contributions to the MIT core are welcome. Open-core boundaries are documented in EDITIONS.md.

Do not open public issues for vulnerabilities. Use GitHub Security Advisories or contact the maintainer privately. The target response window is 48 hours, with a fix target of 14 days for confirmed high-impact issues.

Built by sharvik.tech

TITAN Gateway — govern every token before it leaves your boundary.

About

No description, website, or topics provided.

Resources

Readme

License

MIT, Unknown licenses found

Licenses found

MIT
LICENSE
Unknown
LICENSE-ENTERPRISE.md
Activity

Stars

3 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages