TemporalRAG

A DAG-versioned Retrieval-Augmented Generation (RAG) methodology designed for deterministic LLM query reproducibility across historical knowledge states.

Key Features

• Deterministic Reproducibility: Achieves 100% deterministic query reproducibility across historical knowledge states utilizing dense vector embeddings and a DAG-versioned RAG approach.
• Asynchronous Ingestion Pipeline: Architected with a message broker and worker nodes to process and store immutable SHA-256 snapshots in object storage and PostgreSQL, guaranteeing zero embedding duplication.
• Advanced Retrieval & Re-ranking: Implements a two-stage HNSW retrieval and semantic re-ranking system (utilizing BAAI/bge-reranker-base) optimized for LLaMA-3.
• Scalable Performance: Validated temporal context isolation on 500 arXiv papers, demonstrating linear latency scaling.

Architecture & Tech Stack

• Database & Vector Store: PostgreSQL with pgvector
• Object Storage: MinIO
• Message Broker / Queue: Redis
• Re-ranking Engine: HuggingFace Text Embeddings Inference
• LLM: LLaMA-3

Project Structure

.
├── backend/              # API server and core application logic
├── frontend/             # User interface
├── postgres/             # Database initialization scripts (e.g., schema.sql)
├── worker/               # Worker nodes for asynchronous data ingestion
└── docker-compose.yaml   # Infrastructure orchestration for dependencies

Getting Started

Prerequisites

• Docker and Docker Compose
• Language runtimes (Node.js/Python) depending on specific backend, frontend, and worker implementations.

1. Launch Infrastructure Dependencies

The provided docker-compose.yaml provisions the PostgreSQL vector database, HuggingFace re-ranker, Redis queue, and MinIO object storage.

From the root directory, execute:

docker-compose up -d

Photos

Provisioned Services & Ports:

• PostgreSQL (pgvector): localhost:5432 (User: postgres, Password: password, DB: db)
• HuggingFace Re-ranker: localhost:8080
• Redis Queue: localhost:6379
• MinIO Object Storage: API at localhost:9000 | Console at localhost:9001 (User: user, Password: password)

2. Start Application Services

Navigate to the respective directories to start the application components. (Adjust the package manager commands based on your specific setup).

Database Initialization: The docker-compose.yaml automatically mounts and executes ./postgres/schema.sql on database initialization.

Backend:

cd backend
# Install dependencies and run the server
bun install
bun run dev

Worker:

cd worker
# Install dependencies and start the background ingestion worker
bun install
bun run dev

Frontend:

cd frontend
# Install dependencies and run the client application
bun install
bun run dev