🗂️ AI Document Preprocessor

Convert any office document into clean, LLM-ready Markdown — locally, instantly, privately.

---

Overview

AI Document Preprocessor is a standalone Windows desktop application that converts PDF, DOCX, PPTX, XLSX, HTML, and TXT files into optimised Markdown — ready to paste into ChatGPT, Claude, Gemini, or any LLM prompt.

All processing happens 100% locally. No API keys. No subscriptions. No data leaves your machine.

---

✨ Features

Feature	Description
📄 PDF → Markdown	Native text extraction via PyMuPDF; OCR fallback for scanned documents
📝 DOCX → Markdown	Full structure preservation via python-docx + mammoth
📊 PPTX → Markdown	Slide-by-slide extraction with speaker notes
📈 XLSX → Markdown	Spreadsheet → formatted Markdown tables
🌐 HTML → Markdown	Web page content extraction and cleaning
📃 TXT → Markdown	Direct passthrough with light formatting
🧹 AI Cleaning	Removes noise, normalises headers, fixes list formatting
👁️ Live Preview	Raw / Clean / Rendered three-tab workspace
🔢 Token Counter	Real-time token usage (tiktoken cl100k_base)
📦 Smart Export	Auto-names files, remembers last folder, handles duplicates
🔒 100% Local	Zero telemetry, zero network calls, zero cloud dependency

---

Screenshots

Home — Upload	Converting	Preview

---

📦 Installation

Option A — Installer (Recommended)

1. Download AI-Document-Preprocessor-Setup.exe from the Releases page
2. Double-click the installer and follow the wizard
3. Launch AI Document Preprocessor from the Start menu or Desktop shortcut

Option B — Portable

1. Download AI-Document-Preprocessor-Portable.zip from the Releases page
2. Extract anywhere (USB drive, Desktop, etc.)
3. Run AI-Document-Preprocessor.exe — no installation required

System Requirements

• Windows 10 / 11 (64-bit)
• 4 GB RAM recommended
• ~500 MB disk space

No Python installation required. Everything is bundled in the executable.

---

🚀 Usage

1. Upload a Document

Click Browse or drag-and-drop any supported file onto the upload panel.

Supported formats: .pdf .docx .pptx .xlsx .html .txt

2. Convert

Click Convert and watch the animated progress overlay. Conversion typically takes under 1 second for most documents.

3. Preview

The workspace shows three tabs:

Tab	Contents
Raw	Original extracted Markdown — unchanged output
Clean	AI-optimised Markdown — noise removed, structure fixed
Rendered	Live Markdown preview with syntax highlighting

4. Export

Action	Behaviour
Export	Opens system save dialog, defaults to ~/Downloads, pre-fills filename
Copy Markdown	Copies cleaned Markdown to clipboard
Open Folder	Opens the export folder in Windows Explorer

Auto-generated filenames:

resume.pdf          →  resume_cleaned.md
slides.pptx         →  slides_cleaned.md
report (2).docx     →  report_cleaned (1).md   (if duplicate)

---

🏗️ Architecture

ai-document-preprocessor/
├── app.py                            # Entry point, splash screen, crash handler
├── installer.py                      # GUI installer wizard
├── requirements.txt                  # Python dependencies
├── build_release.bat                 # Full release build pipeline
├── AI-Document-Preprocessor.spec     # PyInstaller packaging configuration
│
├── services/
│   ├── converter.py                  # Format detection + conversion pipeline
│   ├── cleaner.py                    # Markdown cleaning & LLM optimisation
│   ├── enhancer.py                   # Token counting, diagnostics
│   └── cache_manager.py              # Conversion result caching
│
├── ui/
│   ├── main_window.py                # Main orchestrator + async pipeline
│   ├── theme.py                      # Design tokens & colour system
│   └── components/
│       ├── upload_panel.py           # File drop / browse panel
│       ├── preview_panel.py          # Raw/Clean/Rendered tab workspace
│       ├── loading_overlay.py        # Animated conversion overlay
│       └── ...                       # Additional UI components
│
├── assets/                           # Application icons
└── docs/
    └── screenshots/                  # UI screenshots

Conversion Pipeline

File selected
    ↓
Extension-based type detection
    ↓
Format-specific converter
(PdfConverter / DocxConverter / PptxConverter / …)
    ↓
Raw Markdown extraction
    ↓
AI cleaning pass (cleaner.py)
    ↓
Token counting (tiktoken)
    ↓
Render to workspace preview

All heavy work runs in a ThreadPoolExecutor off the UI thread. A 60-second timeout guard prevents indefinite hangs.

---

🛠️ Tech Stack

Layer	Technology
UI Framework	Flet 0.21+ (Flutter-based Python UI)
PDF Extraction	PyMuPDF (fitz)
Document Parsing	MarkItDown (Microsoft)
DOCX	python-docx + mammoth
PPTX	python-pptx
XLSX	openpyxl + pandas
Token Counting	tiktoken (OpenAI)
OCR (optional)	RapidOCR + ONNX Runtime
Packaging	PyInstaller 6.x
CI/CD	GitHub Actions

---

⚠️ Known Limitations

• Windows only in v1.0 (macOS / Linux planned for v1.1)
• Scanned PDFs without embedded text rely on OCR — accuracy varies
• Very large files (>50 MB) may take 10–30 seconds
• Complex XLSX merged cells may lose some structure in conversion
• Password-protected documents are not supported

---

🔮 Roadmap

See ROADMAP.md for planned features including:

• Cross-platform (macOS, Linux)
• Batch / folder processing
• Optional LLM-assisted cleaning
• Additional format support (.eml, .epub, images)
• REST API mode

---

🤝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md to get started.

# Quick start for developers
git clone https://github.com/YOUR_USERNAME/ai-document-preprocessor.git
cd ai-document-preprocessor
pip install -r requirements.txt
python app.py

---

📄 License

This project is licensed under the MIT License — see LICENSE for details.

---

📋 Changelog

See CHANGELOG.md for a full history of changes.

---

Made with ❤️ for the AI / LLM developer community

⭐ Star this repo if you find it useful!