ragservice

A FastAPI service that ingests financial documents over HTTP, chunks and embeds them into a vector index, and answers questions with grounded, cited responses. Project 4 of 6 in a regulated-document AI platform. Agentcli was "the model calls your code," ragservice is "put it behind an API and ground it," dual-backed by Azure AI Search and pgvector.

Status: Phase 5 (grounded generation with citations) — done. Phase 6 (streaming answers) next.

Usage

uv run uvicorn ragservice.main:app --reload

curl -X POST http://localhost:8000/ingest \
  -F "file=@10-K-excerpt.txt" -F "source=Fannie Mae 10-K 2025"

curl -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is the debt-to-income ratio ceiling for this program?"}'

/ask searches the full indexed corpus, not just the document from the most recent /ingest call — retrieval is by relevance to the question, and the response's citations identify which source document actually answered it.

Auth

Authentication follows the same pattern as askdocs and agentcli — secrets are read from environment variables, never hardcoded.

# .env
ENDPOINT=https://your-foundry-project.services.ai.azure.com/openai/v1
MODEL=gpt-4o-mini
TOKENS_MAX=1024

# Phase 2 (embeddings)
EMBEDDING_MODEL=text-embedding-3-small

# Phase 7 (pgvector backend)
RAGSERVICE_BACKEND=azure-search         # or "pgvector"
DATABASE_URL=postgresql://localhost/ragservice

Azure uses DefaultAzureCredential — run az login for local development. No raw API keys in Azure.

Setup

uv sync
az login
uv run uvicorn ragservice.main:app --reload

Development

uv run pytest          # run tests
uv run ruff check src  # lint
uv run pyright src     # type check

Stack

• fastapi — async HTTP API, typed request/response models, dependency injection
• uvicorn — ASGI server
• openai — OpenAI-compatible client pointed at Azure AI Foundry, chat and embeddings
• azure-identity — DefaultAzureCredential for keyless auth
• azure-search-documents — vector + hybrid search index
• pgvector / asyncpg — self-hosted vector backend, second implementation of the retrieval seam
• pytest + httpx — offline ASGI testing with mocked AI seams

What this project adds

• FastAPI replacing Typer as the entry point — the model calls your code, now HTTP calls your code
• Dependency-injected, process-lifetime clients (Depends, lifespan) instead of per-call construction
• Chunking and embedding documents at ingest, batched with bounded concurrency
• A retrieval Protocol implemented twice — Azure AI Search and pgvector — selected by config, not by rewrite
• Grounded generation with citations and a working refusal path for unanswerable questions
• Streaming answers over HTTP (StreamingResponse/SSE) with citations delivered intact
• PII masking at ingest and a documented prompt-injection defense

Part of a series

Project	What it adds
repostat	Python language fundamentals: CLI, REST, typed models, error handling, secrets, tests
askdocs	LLM SDK, streaming, naive RAG, multi-provider
agentcli	Tool-calling agents, memory, asyncio, MCP
ragservice	FastAPI, embeddings, vector + hybrid search, citations, PII handling
extractor	Document intelligence, vision, batch processing, structured validation
evalkit	Evals, observability, cost tracking, tracing, Docker