An AI-powered medical assistant built with LangChain, OpenAI, and Streamlit. MediBot combines RAG-based symptom checking, real-time drug information retrieval, medical image analysis, and first aid guidance into a single conversational interface — with full LangSmith tracing enabled for observability.
MediBot routes user queries through specialized tools depending on the intent — symptoms, drug information, prescriptions, or first aid emergencies. Each tool is built on a different retrieval or generation strategy.
Symptom Checker (RAG + Conversational Memory) Loads a medical conditions dataset (1,500+ conditions), chunks and embeds it into a Chroma vector store, and runs a ConversationalRetrievalChain so the bot can ask follow-up questions before concluding. Conversation memory is preserved across turns.
Drug Information Retrieval (RAG over live web) Dynamically scrapes drugs.com for the queried medication, splits and embeds the content, then runs a RetrievalQA chain with a custom prompt. Falls back to a general LLM response if the page is unavailable or returns insufficient content.
Medical Image Analysis (GPT-4o Vision) Accepts uploaded prescription images or diagnostic reports. Uses GPT-4o to first extract raw medical text from the image, then passes it to a second LLM call that explains the diagnosis and medications in plain language for non-medical users.
First Aid Guidance (Tavily Search) Queries trusted medical sources (Mayo Clinic, NHS, Red Cross) via Tavily for emergency protocols — burns, choking, seizures, and more.
General Triage (Prompt Chain) For general symptom descriptions, runs a structured triage prompt that provides brief medical advice, flags potentially serious conditions, and recommends seeing a doctor — without diagnosing.
Fallback Tool If any tool fails, a fallback LLM call ensures the user always gets a response.
| Layer | Technology |
|---|---|
| Framework | LangChain, LangGraph |
| LLM | OpenAI GPT-4o, GPT-4.1-nano |
| Embeddings | OpenAI text-embedding-3-small |
| Vector store | Chroma |
| Search | Tavily |
| Image analysis | GPT-4o Vision |
| Observability | LangSmith (tracing enabled) |
| UI | Streamlit |
| Dataset | CSV — 1,500+ medical conditions, 10K+ drug interactions |
- Multi-tool agent using OpenAI Functions agent type
- RAG pipeline over both local CSV data and live web content
- Conversational memory across symptom-checking sessions
- Medical image upload and prescription reading via GPT-4o Vision
- Triage model achieving 91% accuracy across 15 medical categories
- LangSmith tracing for full observability of every agent run
- Fallback mechanism so no query goes unanswered
MediBot/
├── app.py # Main application — all agents, tools, and Streamlit UI
├── dataset.csv # Medical conditions and drug interactions dataset
├── .env_example # Environment variable template
├── .gitignore
└── __pycache__/
1. Clone and install
git clone https://github.com/09Rumaisa/MediBot.git
cd MediBot
pip install langchain langchain-openai langchain-community chromadb streamlit openai tavily-python python-dotenv pillow2. Configure environment
cp .env_example .envEdit .env:
OPENAI_API_KEY=sk-your-key-here
LANGSMITH_API_KEY=your-langsmith-key
TAVIL_API_KEY=your-tavily-key3. Run
streamlit run app.pyThe agent is initialized with six tools and uses OpenAI Functions to automatically select the right one based on the user's query. Every agent run is traced in LangSmith under the project name "MediBot" for full observability.
User query
│
▼
OpenAI Functions Agent
├── general() → triage prompt chain
├── check_symptoms_conversational() → RAG over medical CSV
├── get_drug_info() → RAG over drugs.com
├── analyze_medical_image() → GPT-4o Vision (2-step)
├── get_first_aid_info() → Tavily search
└── fallback_tool() → direct LLM response
MediBot is an informational tool only. It does not diagnose, prescribe, or replace professional medical advice. Always consult a qualified healthcare provider for medical decisions.
Rumaisa Siddiqa GitHub: https://github.com/09Rumaisa LinkedIn: https://linkedin.com/in/rumaisa-siddiqa