🛡️ ScamCall Guardian

A Hybrid Ensemble Approach for Real-Time Multilingual Phone Scam Detection in Indian Languages — now with Redis semantic caching, Prometheus/Grafana observability, and Render cloud deployment.

ScamCall Guardian analyses phone call transcripts in English, Hindi, Hinglish, Tamil, and Tanglish to detect scam calls — OTP fraud, fake KYC renewals, "digital arrest" threats, job scams, and more. It provides real-time, in-call warnings with multilingual alerts (English, Hindi, Tamil) using a novel three-layer detection architecture.

📄 Read the full research report →

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    ScamCall Guardian Pipeline                   │
│                                                                 │
│  [Audio / Text Input]                                          │
│       │                                                         │
│       ▼                                                         │
│  Groq Whisper (cloud STT)           [stage: stt]               │
│       │                                                         │
│       ▼                                                         │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │             Scam Detection Pipeline                      │  │
│  │                                                          │  │
│  │  ├── Rules Engine  (regex, 13 categories, <1ms)          │  │
│  │  │   [stage: rules]                                      │  │
│  │  │                                                       │  │
│  │  ├── MuRIL Classifier  (110M params, <100ms)             │  │
│  │  │   [stage: muril]                                      │  │
│  │  │                                                       │  │
│  │  └── LLM Reasoner  (LLaMA 3.3 via Groq, ~2s)            │  │
│  │      [stage: llm]                                        │  │
│  │           │                                              │  │
│  │           ▼   ┌─────────────────────────────────────┐   │  │
│  │     ┌─ Cache? ┤  Redis Stack (HNSW vector index)    │   │  │
│  │     │  HIT◄───┤  paraphrase-multilingual-MiniLM     │   │  │
│  │     │  MISS──►┤  cosine sim ≥ 0.92 → return cached  │   │  │
│  │     │         └─────────────────────────────────────┘   │  │
│  │     │                                                    │  │
│  │     ▼  (on MISS: call LLM → store in Redis, TTL 24h)    │  │
│  │  LLaMA 3.3 (Groq API)                                    │  │
│  │                                                          │  │
│  └──────────────────────────────────────────────────────────┘  │
│       │                                                         │
│       ▼                                                         │
│  Deterministic Scorer (weighted fusion, auditable Python)       │
│       │                                                         │
│       ▼                                                         │
│  Alert Generator (multilingual: English, Hindi, Tamil)          │
│       │                                                         │
│       ▼         ┌──────────────────────────────────────────┐   │
│  /metrics ──────► Prometheus ─────► Grafana Cloud           │   │
│                 └──────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Why cache only the LLM stage?

Stage	Speed	Cacheable?	Reason
Whisper STT	~1–3s	❌ No	Audio bytes are never repeated
MuRIL	<100ms	❌ No	Local inference is fast; caching adds complexity for nothing
LLM (LLaMA 3.3)	~2s	✅ Yes	Deterministic on same text; scam scripts are highly repetitive

Why semantic caching over exact-match?

Scam callers paraphrase constantly: "OTP share karo" vs "verification code batao" vs "apna code dena". An exact-match key would never hit. Cosine similarity at 0.92 catches all three as the same intent while rejecting genuinely different conversations.

Why threshold 0.92?

This is a risk-scoring system. A false cache hit returns the wrong verdict on a live scam call — telling someone a scam is safe. That is worse than a cache miss. 0.92 is strict by design; tune it downward only after measuring your false-hit rate.

📊 Benchmark Results

Run the load test yourself and fill in the real X/Y/Z values below.

# 1. Start the stack
docker compose up -d

# 2. No-cache baseline (X-Bypass-Cache: true header)
python load_test.py --rate 5 --no-cache --output results_nocache.json

# 3. Warm cache run
python load_test.py --rate 5 --output results_cached.json

# 4. Print comparison table
python load_test.py --compare results_nocache.json results_cached.json

Metric	No Cache	Cache (warm)	Improvement
p50 latency (ms)	47,563.3	2,844.5	−94%
p95 latency (ms)	72,853.2	6,640.6	−91%
p99 latency (ms)	76,882.0	6,944.6	−91%
mean latency (ms)	46,960.1	2,808.6	−94%
Cache hit rate	—	70%	—

🎯 The Problem

Phone scams are a ₹100+ crore problem in India. Elderly, rural, and non-English speakers are the most vulnerable. Existing solutions only block numbers after reports — nothing gives a real-time, in-call warning tailored to Indian scam patterns.

📂 Project Structure

scamcall-guardian/
├── backend/
│   ├── main.py                 # FastAPI (REST + WebSocket + /metrics)
│   ├── config.py               # All env-var config (incl. Redis, cache, logging)
│   ├── stt_engine.py           # Groq Whisper STT (full + chunked)
│   ├── rules_engine.py         # Regex-based scam patterns (13 categories)
│   ├── ml_classifier.py        # TF-IDF + LogReg (baseline classifier)
│   ├── transformer_classifier.py # Fine-tuned MuRIL (primary classifier)
│   ├── llm_reasoner.py         # Groq LLaMA (few-shot + CoT + semantic cache)
│   ├── scorer.py               # Deterministic fusion scorer (with stage timing)
│   ├── alert_manager.py        # Bilingual alert builder
│   ├── semantic_cache.py       # [NEW] Redis HNSW semantic cache (Part A)
│   ├── observability.py        # [NEW] Prometheus metrics + structured logging (Part B)
│   ├── requirements.txt
│   └── Dockerfile
├── frontend/
│   ├── index.html              # Web dashboard
│   ├── style.css               # Dark-mode premium UI
│   └── app.js                  # Interactive frontend
├── tests/
│   └── test_smoke.py           # [NEW] Smoke tests (no GPU/API keys needed)
├── grafana/
│   └── provisioning/           # [NEW] Auto-datasource config for local Grafana
├── load_test.py                # [NEW] Async replay load-tester (Part C)
├── render.yaml                 # [NEW] Render Blueprint (IaC for cloud deploy)
├── docker-compose.yml          # Full local stack: app + Redis + Prometheus + Grafana
├── prometheus.yml              # Prometheus scrape config
├── .env.example                # All environment variables documented
├── REPORT.md                   # Research report
└── README.md

🚀 Quick Start (Local)

Prerequisites

Python 3.10+
Docker & Docker Compose
A free Groq API key

1. Clone & Setup

cp .env.example backend/.env
# Edit backend/.env — set GROQ_API_KEY at minimum

2. Run the full stack (app + Redis + Prometheus + Grafana)

docker compose up -d

Service	URL
ScamCall Guardian API	http://localhost:8000
Prometheus	http://localhost:9090
Grafana (local)	http://localhost:3000 (admin / scamguard)
Prometheus metrics	http://localhost:8000/metrics
Metrics summary	http://localhost:8000/api/metrics-summary

3. Run without Docker (development)

cd backend
pip install -r requirements.txt
uvicorn main:app --reload

Requires Redis Stack running locally — see docker-compose.yml redis service.

☁️ Cloud Deployment (Render)

1. Connect repo to Render

Go to render.com → New → Blueprint
Select this repository — Render reads render.yaml automatically
Two services are created: scamguard-backend (Web Service) + scamguard-redis (managed Redis)

2. Set environment variables

In Render dashboard → scamguard-backend → Environment:

GROQ_API_KEY → your Groq API key

All other variables have defaults in render.yaml.

3. Set up Grafana Cloud (free, ~5 minutes)

Sign up at grafana.com → free tier
Connections → Add new connection → Prometheus
URL: https://scamguard-backend.onrender.com/metrics
Import the dashboard panels below

4. CI/CD

Render auto-deploys on every push to main. No GitHub Actions file needed.

📈 Grafana Dashboard Panels

Build these four panels (PromQL queries):

Panel 1 — p50/p95 latency per stage

histogram_quantile(0.95, sum(rate(pipeline_stage_seconds_bucket[5m])) by (le, stage))
histogram_quantile(0.50, sum(rate(pipeline_stage_seconds_bucket[5m])) by (le, stage))

Panel 2 — Cache hit rate

rate(llm_cache_events_total{result="hit"}[5m])
  /
(rate(llm_cache_events_total{result="hit"}[5m]) + rate(llm_cache_events_total{result="miss"}[5m]))

Panel 3 — Token usage (tokens/hour)

sum(increase(llm_tokens_total[1h])) by (direction)

Panel 4 — Requests/min

sum(rate(pipeline_requests_total[1m])) by (status) * 60

TODO: Screenshot your dashboard here after deploying to Render.

🔬 How It Works

Three-Layer Detection

Layer	Method	Speed	Purpose
1. Rules Engine	Regex patterns (13 categories)	< 1ms	Instant detection of known scam keywords
2. MuRIL Classifier	Fine-tuned transformer (110M params)	< 100ms	Deep semantic understanding of multilingual text
3. LLM Reasoner	LLaMA 3.3 via Groq (few-shot + CoT)	~2s → ~5ms cached	Contextual analysis, social-engineering detection

Scoring Formula

final_score = 0.30 × rule_score + 0.35 × ml_score + 0.35 × llm_score

Hard Override: if rule_score ≥ 80 → final_score = max(final, 80)

Verdict:
  > 50  → ⚠️  SCAM DETECTED (danger)
  > 30  → ⚡ SUSPICIOUS (caution)
  ≤ 30  → ✅ LOOKS SAFE

Semantic Cache (Part A)

Cache lookup flow:
  1. Embed transcript with paraphrase-multilingual-MiniLM-L12-v2 (384-dim)
  2. HNSW KNN-1 search in Redis (sub-millisecond ANN lookup)
  3. cosine similarity = 1 − (dist / 2)   [for L2-normalised vectors]
  4. If similarity ≥ 0.92 → return cached result  (HIT)
  5. Else → call LLM → store result with 24h TTL   (MISS)

Observability (Part B)

GET /metrics  →  Prometheus text format
  pipeline_stage_seconds{stage="llm"}   — p50/p95 per stage
  llm_cache_events_total{result="hit"}  — cache hit counter
  llm_tokens_total{direction="prompt"}  — Groq token usage
  pipeline_requests_total{status="ok"}  — request volume

Structured JSON log per request:
  {"ts":"...", "request_id":"a1b2c3", "stage_ms":{"rules":0.4,"muril":80,"llm":5},
   "cache":"hit", "tokens":{"prompt":0,"completion":0}, "risk_score":72.5, ...}

📊 Research Results

Ablation Study (977 test samples)

Configuration	Accuracy	F1 (Macro)	Precision	Recall	AUC-ROC
Rules Only	85.9%	0.541	0.928	0.543	0.741
ML (TF-IDF)	99.1%	0.983	0.972	0.995	0.999
ML (MuRIL)	93.9%	0.894	0.866	0.932	0.966
LLM Only	87.0%	0.622	0.933	0.594	0.532
Rules + TF-IDF	93.8%	0.856	0.966	0.798	0.999
Rules + MuRIL	92.8%	0.833	0.945	0.776	0.975
Full Pipeline	93.2%	0.853	0.917	0.811	0.964

MuRIL fine-tuning metrics (test set):

Accuracy: 93.96% | F1 (macro): 0.894 | Scam recall: 92% | Best val F1: 0.910

🔧 Load Testing

# Run baseline (no cache) — X-Bypass-Cache header skips Redis
python load_test.py --url http://localhost:8000 --rate 5 --no-cache --output results_nocache.json

# Run cache-enabled (warm cache after first pass)
python load_test.py --url http://localhost:8000 --rate 5 --output results_cached.json

# Print comparison table with p50/p95/p99
python load_test.py --compare results_nocache.json results_cached.json

Options:

--rate — requests per second (default: 5)
--limit — max transcripts to send (default: 100)
--url — API base URL (works with Render URL too)
--concurrency — max parallel requests (default: 10)

🛠️ Tech Stack

Layer	Technology
Backend	FastAPI, Python 3.10+, Uvicorn
ML — Transformer	MuRIL (google/muril-base-cased, 110M params), PyTorch, HuggingFace
ML — Baseline	scikit-learn (TF-IDF + Logistic Regression)
LLM	Groq (LLaMA 3.3 70B), few-shot + chain-of-thought
STT	Groq Whisper API (chunked streaming)
Semantic Cache	Redis Stack (RediSearch HNSW), sentence-transformers
Observability	Prometheus, Grafana Cloud
Frontend	Vanilla HTML/CSS/JS, Web Audio API, WebSocket
Deployment	Render (Web Service + managed Redis), Docker Compose
Testing	pytest, FastAPI TestClient

🔮 Resume Bullet

Reduced p95 LLM-stage latency from 72.8s → 6.6s via Redis semantic caching (HNSW vector lookup, 70% hit rate on repetitive scam scripts); instrumented per-stage latency, token, and cost observability with Prometheus/Grafana Cloud; deployed on Render via Docker + render.yaml Blueprint IaC.

🔮 Future Enhancements

Fine-tune IndicBERT / regional models for Tamil, Telugu, Bengali
Android app with call-screening API integration
End-to-end streaming with Whisper's streaming mode
Government integration with TRAI DND registry and 1930 helpline
Adversarial robustness testing against adaptive scammers

📄 License

MIT License — feel free to use, modify, and distribute.

📖 Citation

@misc{scamcall-guardian-2026,
  title={ScamCall Guardian: A Hybrid Ensemble Approach for Real-Time
         Multilingual Phone Scam Detection in Indian Languages},
  author={S Jaya Pradeep},
  year={2026},
  howpublished={\url{https://github.com/JPisOP007/ScamGuard}}
}

ScamCall Guardian — Protecting India from phone scams, one call at a time 🇮🇳

Navigation Menu

Repository files navigation

🛡️ ScamCall Guardian

🏗️ Architecture

Why cache only the LLM stage?

Why semantic caching over exact-match?

Why threshold 0.92?

📊 Benchmark Results

🎯 The Problem

📂 Project Structure

🚀 Quick Start (Local)

Prerequisites

1. Clone & Setup

2. Run the full stack (app + Redis + Prometheus + Grafana)

3. Run without Docker (development)

☁️ Cloud Deployment (Render)

1. Connect repo to Render

2. Set environment variables

3. Set up Grafana Cloud (free, ~5 minutes)

4. CI/CD

📈 Grafana Dashboard Panels

🔬 How It Works

Three-Layer Detection

Scoring Formula

Semantic Cache (Part A)

Observability (Part B)

📊 Research Results

Ablation Study (977 test samples)

🔧 Load Testing

🛠️ Tech Stack

🔮 Resume Bullet

🔮 Future Enhancements

📄 License

📖 Citation

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
backend		backend
frontend		frontend
grafana/provisioning/datasources		grafana/provisioning/datasources
tests		tests
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
REPORT.md		REPORT.md
docker-compose.yml		docker-compose.yml
load_test.py		load_test.py
prometheus.yml		prometheus.yml
render.yaml		render.yaml

Folders and files

Latest commit

History

Repository files navigation

🛡️ ScamCall Guardian

🏗️ Architecture

Why cache only the LLM stage?

Why semantic caching over exact-match?

Why threshold 0.92?

📊 Benchmark Results

🎯 The Problem

📂 Project Structure

🚀 Quick Start (Local)

Prerequisites

1. Clone & Setup

2. Run the full stack (app + Redis + Prometheus + Grafana)

3. Run without Docker (development)

☁️ Cloud Deployment (Render)

1. Connect repo to Render

2. Set environment variables

3. Set up Grafana Cloud (free, ~5 minutes)

4. CI/CD

📈 Grafana Dashboard Panels

🔬 How It Works

Three-Layer Detection

Scoring Formula

Semantic Cache (Part A)

Observability (Part B)

📊 Research Results

Ablation Study (977 test samples)

🔧 Load Testing

🛠️ Tech Stack

🔮 Resume Bullet

🔮 Future Enhancements

📄 License

📖 Citation

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages