ZeroCost-Codebase-RAG Agentic AI

Hybrid Graph RAG + Report RAG System

A production-ready Retrieval-Augmented Generation engine that combines three retrieval paths — vector search, knowledge graph traversal, and community report summarization — to deliver deep, relationship-aware answers about your codebase.

The Pitch: Traditional RAG only finds similar chunks. Our hybrid system understands how code connects — which functions call which, what classes inherit from what, how subsystems form communities — and answers architectural questions that chunk-only RAG simply cannot.

---

Architecture Overview

Three-Path Hybrid Retrieval

Query -> Classifier -> Parallel Retrieval -> RRF Fusion -> LLM Synthesis
                          |
          +---------------+---------------+
          v               v               v
     Path 1: Vector    Path 2: Graph    Path 3: Report
     (chunk search)    (entity + k-hop) (community summaries)

Path	Engine	What It Finds
Vector	BM25 + cross-encoder reranker + HyDE expansion	Relevant code/text chunks
Graph	Entity extraction -> k-hop subgraph expansion	Functions, classes, imports, call relationships
Report	Community detection -> LLM-generated summaries	Subsystem-level architectural overviews

Knowledge Graph Pipeline (Graph RAG)

Codebase -> PyExtractor (AST) + PolyglotExtractor (regex)
         -> NetworkX DiGraph (nodes: file/function/class/endpoint)
         -> Community Detection (Leiden algorithm)
         -> LLM Community Reports (~200-word subsystem summaries)

Query Flow

1. Classify — Keyword-based classification: code-structure, conceptual, architectural, general
2. Parallel Retrieve — All relevant paths run concurrently via ThreadPoolExecutor
3. RRF Fusion — Reciprocal Rank Fusion (K=60) merges ranked results across paths
4. LLM Synthesize — Token-budget-aware prompt with all three context sources fed to local Ollama

---

Setup & Installation

Prerequisites

• Python 3.10+
• Ollama running locally (ollama serve)
• Pull the model: ollama run llama3.2:3b

Or API keys

• API key
• Model name

Install Dependencies

pip install -r requirements.txt

Run CLI Interface

python main.py # For Terminal view
python web_app.py # For web interface

# Double click - start.bat to run without opening terminal
start.bat

Enter the absolute path to your codebase folder. The system will automatically:

1. Ingest all files into the vector store (delta-aware — re-runs are instant)
2. Build the knowledge graph (AST + regex extraction -> NetworkX DiGraph)
3. Detect communities and generate LLM reports per subsystem
4. Start interactive Q&A using all 3 retrieval paths with RRF fusion

Run Web Interface

Easiest (Windows): double-click start.bat — it starts the server (using your venv) and opens http://localhost:8000 in the browser automatically.

Manually:

python web_app.py

Open http://localhost:8000 in your browser.

Web Interface Features:

• Dark animated UI with real-time step-by-step progress
• Drag-and-drop folder upload or manual path input
• 4-step indexing pipeline: Vector -> Graph -> Reports -> Orchestrator
• Streaming token-by-token answers via SSE
• "Thinking..." and "Agent..." status indicators during queries
• Interactive chat with message history

LLM Providers (Model Settings):

• Click the ⚙ Model button in the header to choose which LLM powers answers
• Works fully offline with Ollama by default — no API key needed
• Ollama model picker — the model list is read live from your local Ollama install, so any pulled model (e.g. qwen2.5:14b) is selectable; or type any model name
• Optional cloud providers — add a key in the web UI to switch:
  • OpenAI GPT (gpt-4o-mini)
  • NVIDIA NIM (meta/llama-3.3-70b-instruct)
  • Google Gemini (gemini-2.0-flash)
  • Anthropic Claude (claude-sonnet-4-20250514)
• Custom API URL per provider — each provider has an editable API URL field (pre-filled with the correct default). Point it at any OpenAI-compatible endpoint or a private NIM deployment
• Override the default model per provider; leave fields blank to re-use already-saved values (masked key shown)
• Switch back to local Ollama anytime (no key required)
• Keys are stored server-side in db_storage/provider_config.json (gitignored — never committed)

Session Management:

• New Chat — start fresh with any folder, anytime
• Session History — dropdown lists every indexed folder (named after the folder)
• Instant Resume — reopening a session loads cached data instantly; chat is usable while a delta check runs in the background
• Background Updates — live progress shown in the step cards (Vector -> Graph -> Reports -> Orchestrator); when done, the orchestrator is swapped with fresh data
• Delta-Aware — mtimes tracked per relative path, so duplicate filenames in subfolders don't trigger false rebuilds; only changed files are re-processed

API Endpoints:

Method	Endpoint	Description
GET	/	Chat UI
GET	/api/sessions	List indexed sessions (chat history)
POST	/api/init	{"path": "..."} — resume returns JSON instantly + starts background update; fresh index streams SSE progress
GET	/api/update-status	Background update progress (running, progress[], done, error)
POST	/api/upload	Upload files (multipart), returns temp path
POST	/api/query	{"query": "..."} — SSE stream of status + token-by-token answer
GET	/api/status	Check if system is initialized
GET	/api/providers	List LLM providers + masked API keys + active provider
POST	/api/providers	{"provider", "api_key", "model", "base"} — save key/model/URL + activate provider ("ollama" returns to local)
GET	/api/ollama-models	Models pulled in your local Ollama install (for the model picker)

Session data is persisted under db_storage/:

• sessions.json — registry (session name, path, last indexed time)
• persist_<session_id>/graph.json — cached knowledge graph
• persist_<session_id>/reports.json — cached community reports
• persist_<session_id>/meta.json + embeddings.npy — vector store (delta-aware)

---

Project Structure

New_RAG_System/
├── config.py                  # All settings: Ollama, context math, Graph RAG params
├── main.py                    # CLI entry point (interactive Q&A loop)
├── orchestrator.py            # Hybrid orchestrator (classify -> 3-path -> RRF -> synthesize)
├── rag_agent.py               # SessionRAGAgent (HyDE expansion, map-reduce)
├── vector_store.py            # Persistent vector DB (BM25 + cross-encoder reranker)
├── document_loader.py         # Polyglot parsing: PDF, DOCX, code files
├── embeddings.py              # Local sentence-transformers (all-mpnet-base-v2)
├── tree_builder.py            # RAPTOR hierarchical file summaries
├── codebase_analyzer.py       # Polyglot AST/regex analyzer
├── hardware_monitor.py        # GPU VRAM profiler (auto-detects concurrency limits)
├── web_app.py                 # FastAPI web interface (SSE streaming, file upload)
├── start.bat                  # Windows one-click launcher (server + browser)
├── templates/
│   └── index.html             # Browser chat UI (dark theme, animated)
├── graph_rag/
│   ├── __init__.py
│   ├── knowledge_graph.py     # AST + regex extractors -> NetworkX DiGraph
│   └── graph_retriever.py     # Entity extraction + k-hop subgraph retrieval
├── report_rag/
│   ├── __init__.py
│   ├── community_detector.py  # Leiden/greedy modularity community detection
│   └── report_indexer.py      # LLM community report generation with factsheets
├── templates/
│   └── index.html             # Browser chat UI (dark theme, animated)
├── db_storage/                # Session persistence (gitignored)
│   ├── sessions.json          # Session registry (name, path, last indexed)
│   └── persist_<id>/          # Per-session: graph, reports, vector store
└── requirements.txt

---

Configuration (config.py)

Setting	Default	Purpose
MAX_HOPS	2	Subgraph expansion depth for graph retrieval
COMMUNITY_RESOLUTION	1.0	Granularity of community detection
MIN_COMMUNITY_SIZE	5	Ignore communities smaller than this
RRF_K	60	Reciprocal Rank Fusion constant
GRAPH_RAG_ENABLED	True	Toggle Path 2 (graph retrieval)
REPORT_RAG_ENABLED	True	Toggle Path 3 (report retrieval)
CHUNK_SIZE	500	Token size for code/text chunks
MAX_SAFE_TOKENS	dynamic	Auto-computed from num_ctx with 15% safety buffer

---

Why Hybrid?

Chunk-only RAG fails on questions like:

"What subsystems depend on the authentication module?" "Trace the call chain from the API endpoint to the database query."

Our hybrid system solves this by:

• Graph path — traverses call/import/inheritance edges to find structural answers
• Report path — provides pre-computed subsystem summaries for architectural overview
• Vector path — handles keyword/semantic matches the other paths might miss
• RRF Fusion — merges all three into a single ranked context, avoiding context-window waste

Everything runs 100% local by default — no cloud APIs, no data leakage, zero cost per query. Optional cloud LLM providers (OpenAI, NVIDIA NIM, Gemini, Anthropic) can be enabled per-user via the web interface; only your prompt + retrieved context are sent when a cloud provider is selected, never your codebase index.

---

In Action

Web Interface

Backend Processing (4-Step Pipeline)

Proof of Response

---

Built With

• sentence-transformers — Local embeddings (all-mpnet-base-v2) + cross-encoder reranker
• NetworkX — Knowledge graph (DiGraph) with AST and regex extractors
• Leiden Algorithm — Community detection for subsystem identification
• Ollama — Local LLM inference (llama3.2:3b)
• FastAPI + SSE — Web interface with streaming responses
• BM25 + Dense Hybrid — Combined sparse + dense retrieval with Reciprocal Rank Fusion