Skip to content

Latest commit

 

History

18 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

URLLM

CI PyPI Python 3.11+ License: MIT

Point it at any URL. Get a grounded GDPR & security audit in seconds.

URLLM deterministically extracts a web page's full technical and privacy fingerprint — scripts, cookies, CSP, third-party domains, PII forms, fingerprinting signals, tracking pixels, security headers — then hands the structured data to an LLM for a rigorous compliance and security review. No guessing. No raw HTML dumped into a prompt.

$ urllm https://example-shop.com --deep-dive -o report.md --save-sources ./sources/
╭──────────────────────────────────────────────╮
│ URLLM v0.6.1  GDPR & Security Audit          │
│ Target: https://example-shop.com             │
╰──────────────────────────────────────────────╯

Compliance Quick-Glance
- ❌ No Consent Management Platform detected
- ✅ Privacy Policy link found
- ⚠️  3 cookie(s) without Secure flag
- ⚠️  Tracking pixels from: pixel.tracker.example
- ⚠️  2 third-party domain(s) flagged as non-EU

  Page HTML:      sources/example-shop.com_page.html
  HTTP Headers:   sources/example-shop.com_headers.json   ← full untruncated CSP here
  Footprint JSON: sources/example-shop.com_footprint.json

Querying gemini/gemini-2.5-flash …
Running deep-dive evidence review …

Report saved to report.md

Why URLLM?

Most "AI website audits" dump raw HTML into a prompt and hope for the best. URLLM is different:

  • Deterministic first — extraction is pure Python, reproducible, no hallucinations about what the page contains
  • LLM second — the model reasons over structured JSON, not markup soup
  • Grounded citations — every finding references an actual footprint field
  • Anti-hallucination deep-dive — a second adversarial pass stress-tests the initial findings, separates confirmed facts from inferences, and flags what can't be determined from static analysis

Caution

IMPORTANT: urllm MUST NOT be used as legal advice — this is an automated technical assessment aid supported by genAI. Involve qualified legal counsel for compliance decisions.


Install

# Run instantly with uv (no pip, no venv)
uv run urllm.py https://example.com

# Or install as a persistent CLI tool
uv tool install .
urllm https://example.com

uv is a fast Python package manager. Install it with: curl -LsSf https://astral.sh/uv/install.sh | sh


Usage

urllm <URL> [OPTIONS]

  -m, --model MODEL      LiteLLM model string
                         (default: $LLM_MODEL or gemini/gemini-2.5-flash)
  -o, --output FILE      Write full audit report to a Markdown file
  -v, --verbose          Show where each finding was discovered
                         (which header, tag, or script it came from)
  --deep-dive            Run a second adversarial pass on every 🔴/🟠 finding:
                         evidence-grounded, confidence-rated, concrete fixes
  --save-sources DIR     Save raw page HTML, full HTTP headers, and footprint
                         JSON to DIR (created if absent)
  --json                 Print raw footprint JSON to stdout and exit
                         (no LLM call needed; status output goes to stderr,
                         so the JSON pipes cleanly into jq & friends)
  --fail-on SEVERITY     Exit with code 2 if any deterministic finding is at
                         or above SEVERITY: low | medium | high | critical.
                         Rule-based, no API key needed — built for CI/CD gates.
  --timeout SECONDS      HTTP timeout (default: 15)

Exit codes

Code Meaning
0 Success (and --fail-on threshold not reached, if set)
1 Operational error (fetch failed, timeout, DNS, …)
2 --fail-on threshold reached — at least one finding at or above the given severity

The --fail-on gate uses only deterministic, rule-based findings (never LLM output), so it is reproducible and needs no API key:

Severity Findings
critical No HTTPS; password form over plain HTTP; exposed secret in page source (API keys, private keys)
high Trackers/tracking pixels without a CMP; mixed content; no privacy policy link; CSP that doesn't restrict scripts (unsafe-inline/eval/wildcard); weak TLS version; expired certificate
medium Cookies without Secure; fingerprinting; missing CSP / HSTS / X-Content-Type-Options; CSP in Report-Only mode; missing Subresource Integrity; certificate expiring within 30 days; short HSTS max-age
low Forms collecting PII; version disclosure in Server / X-Powered-By; no security.txt (RFC 9116)

Examples

# Quick audit — console output only
urllm https://example.com

# Full report with Claude, deep-dive review, and all sources saved
urllm https://example.com \
  -m anthropic/claude-sonnet-5 \
  -o audit.md \
  --deep-dive \
  --save-sources ./sources/

# Show exactly where each domain was found (script tag, CSP header, etc.)
urllm https://example.com -v

# Footprint only — no LLM, no API key needed
urllm https://example.com --json

# Pipe JSON into jq — find all non-EU third parties
urllm https://example.com --json 2>/dev/null \
  | jq '.third_parties[] | select(.is_non_eu)'

# CI/CD compliance gate — fail the pipeline on any high or critical finding
urllm https://staging.example.com --json --fail-on high > footprint.json

# Use any LiteLLM-supported model
urllm https://example.com -m gpt-4o
urllm https://example.com -m ollama/llama3.2

What gets extracted

Third parties & CSP

URLLM finds third-party domains from four sources, each tracked separately:

Source Example
<script src="..."> analytics.google.com via script-src
<iframe src="..."> www.youtube.com via iframe embed
<link rel="preconnect"> fonts.googleapis.com via dns-prefetch
CSP header www.jsctool.com via CSP:script-src

The CSP source is the most valuable — it reveals domains that are allowed to run scripts even if they're not in the current page load. With --verbose, each domain shows its exact source in the report.

GDPR & privacy signals

Signal What's checked
Cookies Secure, HttpOnly, SameSite, expiry, first/third-party
Consent platforms 18+ CMPs: Cookiebot, OneTrust, Usercentrics, Didomi, IAB TCF, …
Tracking pixels 1×1 images and <noscript> fallback beacons
Fingerprinting Canvas, WebGL, AudioContext, WebRTC, battery API, hardware probes
Client-side storage localStorage, sessionStorage, IndexedDB, CacheStorage
PII in forms Email, phone, name, address, DOB, government ID, payment card, …
Legal links Privacy policy, Impressum, cookie policy, terms, opt-out notice

Security signals

Signal What's checked
TLS Version (weak < 1.2), certificate issuer, expiry (expired / expiring soon)
Security headers 10 OWASP headers: CSP, HSTS (+ max-age length), X-Frame-Options, Referrer-Policy, COOP, COEP, CORP, …
CSP quality Deterministic parse: unsafe-inline, unsafe-eval, wildcard sources, missing object-src / base-uri / frame-ancestors, Report-Only mode — decorative vs. effective CSPs
Exposed secrets High-confidence scan of page source: Stripe / AWS / Google / GitHub / Slack / OpenAI keys, PEM private-key blocks (redacted in output)
Subresource Integrity Cross-origin <script> / <link> loaded without an integrity hash (supply-chain risk)
Version disclosure Version strings leaked in Server / X-Powered-By headers
security.txt RFC 9116 machine-readable security contact policy
Mixed content HTTP resources on HTTPS pages
Form security Cross-origin submissions, password fields, file uploads

Audit report structure

The LLM produces a structured six-section report:

  1. Tech Stack — frameworks, CMS, bundler fingerprints from script/CSS paths
  2. Data Flow & Third-Party Consumers — every domain classified by role
  3. GDPR Compliance Assessment
    • Lawful basis & consent (CMP, pre-consent loading, cookie attributes)
    • Data minimisation (PII forms, hidden fields)
    • International transfers (Art. 44–49, SCCs, adequacy)
    • Transparency (Privacy Policy, Impressum, Cookie Policy)
    • Fingerprinting & tracking (ePrivacy / TTDSG § 25)
  4. Security Assessment
    • Transport security (TLS, HSTS, mixed content)
    • Security headers per-header ✅/❌
    • Application security (CSRF, CSP effectiveness, credential exposure)
    • Overall posture rating 🔴/🟠/🟡/🟢
  5. Risk Summary Table — all findings sorted by severity
  6. Key Recommendations — top 5, prioritised

With --deep-dive

A second adversarial pass re-examines every 🔴 Critical and 🟠 High finding:

Rating Meaning LLM must provide
✅ Confirmed Direct footprint field + value Concrete fix with config/code example
⚠️ Inferred Plausible but not proven What additional evidence would confirm it
❓ Unverifiable Can't determine from static HTML Specific human investigation steps

Unknown domains are forbidden from speculation — the model must state "requires WHOIS lookup / network traffic analysis" rather than guessing.


Regulatory coverage

Framework Scope
GDPR (EU 2016/679) Art. 5, 6, 13–14, 44–49
ePrivacy Directive (2002/58/EC) Cookie consent, tracking
TTDSG (Germany) § 25 — consent for non-essential device storage
TMG / DDG (Germany) § 5 — Impressum obligation

LLM setup

URLLM talks to LLMs through LiteLLM, so any provider LiteLLM supports works — you only need to supply two things:

  1. A model string — pick it with -m/--model, or set a default via LLM_MODEL. The default is gemini/gemini-2.5-flash.
  2. The matching API key — as an environment variable named by the provider (GEMINI_API_KEY, ANTHROPIC_API_KEY, …). Local models (Ollama) need no key.

The model string is always provider/model-name. The provider prefix tells LiteLLM which API to call and which key to read — get those two aligned and you're done.

No key needed for the deterministic path. --json and --fail-on run entirely offline (pure-Python extraction, no LLM call), so CI gates work without any API key. A key is only required for the narrated audit report.

Provider examples

Google Gemini (default — fast and inexpensive)

export GEMINI_API_KEY="AIza..."
urllm https://example.com                              # uses the default gemini/gemini-2.5-flash
urllm https://example.com -m gemini/gemini-2.5-pro     # deeper reasoning

Anthropic Claude

export ANTHROPIC_API_KEY="sk-ant-..."
urllm https://example.com -m anthropic/claude-sonnet-5
urllm https://example.com -m anthropic/claude-opus-5   # highest quality

OpenAI

export OPENAI_API_KEY="sk-..."
urllm https://example.com -m gpt-4o
urllm https://example.com -m gpt-4o-mini                # cheaper

Local / self-hosted (Ollama) — no API key, nothing leaves your machine

ollama pull llama3.2
urllm https://example.com -m ollama/llama3.2
# Custom host? point LiteLLM at it:
export OLLAMA_API_BASE="http://192.168.1.50:11434"
urllm https://example.com -m ollama/llama3.2

Azure OpenAI

export AZURE_API_KEY="..."
export AZURE_API_BASE="https://your-resource.openai.azure.com"
export AZURE_API_VERSION="2024-02-15-preview"
urllm https://example.com -m azure/your-deployment-name

Set a default model

To avoid passing -m every time, export LLM_MODEL (an explicit -m on the command line still wins):

export LLM_MODEL="anthropic/claude-sonnet-5"
urllm https://example.com          # now uses Claude by default

Put the export lines in your ~/.bashrc / ~/.zshrc (or a .env you source) to make them stick.

Compatibility note: newer models (e.g. Claude Opus 4.7+ / Fable 5) reject sampling parameters. URLLM detects this and automatically retries without temperature, so every provider keeps working — you don't need to configure anything.

For the full list of provider prefixes, key names, and model IDs, see the LiteLLM provider docs.


Limitations

  • Static analysis only — server-rendered HTML only. JavaScript-heavy SPAs will be partially visible. Pair with a headless browser for full SPA coverage.
  • Server-side cookies only — cookies set via document.cookie after page load are not captured.
  • Curated tracker database — ~70 known domains covering the most common EU-market trackers. Unknown domains are flagged as "unknown" for LLM classification.
  • Not legal advice — this is a technical assessment aid. Involve qualified legal counsel for compliance decisions.

Development

# Run the test suite (no API key or network access needed —
# tests run against a local HTTP fixture server, LLM calls are stubbed)
uv run pytest

# With coverage
uv run pytest --cov=urllm --cov-report=term-missing

Architectural decisions are documented in docs/architecture/.


License

MIT

About

privacy and security audit for every URL. Deterministic extraction first, LLM then, with --fail-on-Gate for CI/CD

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages