StudyBot 📚

A Retrieval-Augmented Study Assistant for Grounded Q&A

StudyBot is a Retrieval-Augmented Generation (RAG) application that turns personal course notes into a searchable knowledge base and answers questions using only the retrieved material.

Instead of relying solely on an LLM's internal knowledge, StudyBot retrieves relevant passages from the user's notes, injects them into the generation step, cites the supporting sources, and refuses to answer when the available context is insufficient.

The project focuses on a practical AI engineering problem: how to make LLM-powered applications more grounded, traceable, and reliable.

---

✨ Key Features

• 📄 Ingests .txt, .md, and .pdf study materials
• 🔍 Performs semantic search using local sentence-transformer embeddings
• 🧠 Uses Retrieval-Augmented Generation to ground answers in user-provided notes
• 🤖 Supports Google Gemini for natural-language answer generation
• 💻 Includes a fully local placeholder generator for reproducible, API-free execution
• 📚 Returns source references alongside generated answers
• 🛡️ Refuses unsupported questions when retrieval confidence is too low
• 🔄 Supports a swappable generator architecture for different LLM providers
• 📝 Logs queries, retrieved passages, answers, and errors
• 🧪 Includes automated reliability and consistency evaluation

---

🏗️ Architecture

StudyBot separates document ingestion from question answering.

flowchart LR
    A[Course Notes] --> B[Document Loader]
    B --> C[Chunker]
    C --> D[Sentence Transformer]
    D --> E[(Vector Store)]

    F[User Question] --> G[Input Guardrails]
    G --> H[Question Embedding]
    H --> I[Retriever]

    E --> I

    I --> J{Relevant Context?}
    J -- No --> K[Refuse to Answer]
    J -- Yes --> L[Prompt Builder]
    L --> M[Generator / Gemini]
    M --> N[Grounded Answer + Sources]

    N --> O[Logging & Evaluation]

Loading

Ingestion Pipeline

Documents
   ↓
Document Loader
   ↓
Text Chunking
   ↓
Sentence-Transformer Embeddings
   ↓
Vector Store

Query Pipeline

Question
   ↓
Input Validation
   ↓
Question Embedding
   ↓
Semantic Retrieval
   ↓
Relevance Guardrail
   ↓
Prompt + Retrieved Context
   ↓
LLM / Placeholder Generator
   ↓
Grounded Answer + Sources

A more detailed architecture description is available in docs/architecture.md.

---

🧠 How It Works

1. Document Ingestion

StudyBot reads course notes and converts them into clean text.

Documents are divided into overlapping chunks so relevant information can be retrieved without requiring the entire document to fit into the model context.

2. Embedding

Each chunk is converted into a vector representation using a local sentence-transformer model.

The embeddings and their corresponding text passages are stored locally.

3. Retrieval

When a user asks a question, StudyBot embeds the question and compares it with the stored document embeddings using semantic similarity.

The most relevant passages are retrieved.

4. Guardrails

Before generation, StudyBot evaluates retrieval quality.

If the retrieved material is not sufficiently relevant, the system refuses to answer instead of allowing the language model to guess.

5. Answer Generation

Relevant passages are injected directly into the generation prompt.

StudyBot supports:

• Placeholder generator — fully local and deterministic; returns retrieved passages directly.
• Google Gemini — generates natural-language answers grounded in retrieved context.

6. Source Attribution

Answers include references to the passages used during generation so users can trace the response back to the original notes.

---

🛡️ Reliability & Guardrails

StudyBot is designed around a simple principle:

When the source material does not contain the answer, refusing is better than hallucinating.

The system includes:

• Retrieval relevance thresholds
• Off-topic question detection
• Source attribution
• Deterministic evaluation
• Repeatability checks
• Query and response logging
• Retry/backoff handling for external LLM APIs

The included reliability suite currently passes all 5/5 predefined checks, including an off-topic refusal test, with consistent output across repeated deterministic runs.

---

💬 Example

Question

What is the time complexity of binary search?

Retrieved Context

Binary search repeatedly divides a sorted search interval in half...
Its time complexity is O(log n).

Answer

Binary search has a time complexity of O(log n) because the search
space is reduced by half during each iteration.

Source: sample.md#0

---

Unsupported Question

What is the capital of France?

If the uploaded notes contain no relevant information, StudyBot responds:

I couldn't find anything relevant in your notes to answer that.
Try rephrasing, or add notes on this topic.

This prevents unsupported general-purpose answers from bypassing the retrieval layer.

---

🛠️ Tech Stack

Area	Technology
Language	Python
Retrieval	Semantic Search
Embeddings	Sentence Transformers
Vector Storage	NumPy
LLM	Google Gemini
Document Processing	PyPDF
Configuration	python-dotenv
Testing	Pytest
Architecture	Modular RAG Pipeline

---

📁 Project Structure

studybot-rag-assistant/
│
├── data/
│   └── notes/              # Source documents
│
├── docs/
│   └── architecture.md     # Detailed system architecture
│
├── scripts/
│   ├── ingest.py           # Build the searchable index
│   ├── ask.py              # Ask questions
│   └── evaluate.py         # Run reliability evaluation
│
├── src/                    # Core RAG components
│
├── tests/                  # Automated tests
│
├── .env.example            # Configuration template
├── model_card.md           # Responsible-AI documentation
├── requirements.txt
└── README.md

---

🚀 Getting Started

Prerequisites

• Python 3.10+
• Git

Gemini is optional. The complete retrieval pipeline can run locally without an API key.

1. Clone the Repository

git clone https://github.com/Gravity-2010/studybot-rag-assistant.git
cd studybot-rag-assistant

2. Create a Virtual Environment

Linux / macOS

python -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate

3. Install Dependencies

pip install -r requirements.txt

4. Configure Environment Variables

cp .env.example .env

This step is optional when using the default local generator.

---

📚 Add Your Own Notes

Place .txt, .md, or .pdf files inside:

data/notes/

A sample document is included for testing.

Build the searchable index:

python -m scripts.ingest

---

❓ Ask Questions

Ask a question directly:

python -m scripts.ask "How does binary search work?"

Or launch the interactive prompt:

python -m scripts.ask

---

🤖 Enable Gemini

Add the following to your .env file:

GEN_PROVIDER=gemini
GEMINI_API_KEY=your_api_key_here

Then run the normal query command:

python -m scripts.ask "How does binary search work?"

The generator interface is intentionally modular, allowing additional LLM providers to be added without changing the retrieval pipeline.

---

🧪 Evaluation

Run the reliability suite with:

python -m scripts.evaluate

The evaluator tests both supported and unsupported questions and checks the retrieval and guardrail behavior.

Example:

RELIABILITY REPORT
============================================================
[PASS] How does binary search work?
[PASS] What is the time complexity of binary search?
[PASS] What is the time complexity of bubble sort?
[PASS] How does bubble sort work?
[PASS] Unsupported question correctly refused
------------------------------------------------------------
Passed: 5/5
Consistency: 100%

The evaluator intentionally uses the deterministic local generator so evaluation does not depend on external API availability or rate limits.

---

⚙️ Key Engineering Decisions

Local Embeddings

Sentence-transformer embeddings run locally, avoiding per-query embedding API costs and allowing the retrieval pipeline to operate offline.

Lightweight Vector Store

The project uses a NumPy-based vector store rather than requiring an external vector database.

This keeps the system easy to run locally while preserving an interface that can later be replaced with systems such as FAISS or Chroma for larger datasets.

Swappable LLM Layer

Answer generation is isolated behind a generator interface.

This keeps retrieval, evaluation, and guardrail logic independent from the specific language model provider.

Refuse Rather Than Guess

Low-quality retrieval triggers a refusal instead of allowing an LLM to generate an unsupported answer.

Deterministic Evaluation

Reliability testing uses the local generator rather than a live LLM API, making evaluation reproducible and avoiding failures caused by API quotas or model variability.

---

🔬 Limitations & Future Work

Current limitations include:

• The NumPy vector store is designed for small-to-medium personal document collections rather than millions of documents.
• Retrieval quality depends on chunking strategy and embedding quality.
• Generated answers depend on the capabilities and availability of the configured LLM.
• Evaluation currently uses a small predefined reliability set.

Potential future improvements:

• FAISS or Chroma integration
• Reranking retrieved passages
• Expanded retrieval evaluation metrics
• Additional LLM providers
• Web-based user interface
• Conversation history
• Larger evaluation datasets

---

🤝 Responsible AI

StudyBot is designed to prioritize grounded responses and transparent failure behavior.

The repository includes model_card.md, which documents system behavior, limitations, responsible-AI considerations, and lessons learned during development.

---

👩‍💻 Author

Garvita Jain

M.S. Computer Science — University of Maryland, Baltimore County Software Engineer | AI/ML & Backend Systems

GitHub · LinkedIn