☎️ AI Voice Agent for Intelligent Call Delegation and Context-Aware Call Handling for Android

A production-grade prototype for handling real phone calls with a fully local AI pipeline. No cloud AI APIs (OpenAI/Anthropic) are used for the core logic, ensuring 100% privacy and zero per-call costs for inference.

---

🚀 Key Features

• Local Inference: Runs STT (Whisper), LLM (Llama 3.2 1B), and TTS (Piper) entirely on your local machine.
• Smart Turn-Taking: Energy-based VAD (Voice Activity Detection) handles natural conversation flow.
• Barge-In Support: The AI stops talking immediately if the caller interrupts.
• Advanced Memory:
  • Ground Truth: Authority context for factual accuracy.
  • Profile Memory: Remembers names, facts, and preferences across calls.
  • Semantic Context: Retrieves relevant past conversation snippets.
• Telephony: Seamless integration with Twilio Voice via Media Streams.

---

🛠️ Tech Stack

• Backend: FastAPI (Python 3.10+)
• STT: faster-whisper
• LLM: Ollama (Llama 3.2 1B)
• TTS: Piper
• Audio: NumPy, Scipy (Real-time µ-law/PCM processing)

---

⚡ Quick Start

1. Prerequisites

• Python 3.10+
• Ollama: Download & Install
  • After installation, run: ollama pull llama3.2:1b
• Piper:
  • Download piper.exe from Piper Releases.
  • Place it in models/piper.exe.
  • Download a voice model (e.g., en_US-lessac-medium.onnx) into models/.

2. Installation

# Clone the repository
cd bangaya-v7-best

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

3. Configuration

1. Copy .env.example to .env.
2. Fill in your Twilio Credentials:
   • TWILIO_ACCOUNT_SID
   • TWILIO_AUTH_TOKEN
3. (Optional) Set PUBLIC_HOST to your ngrok URL for reliable connectivity.

4. Running the Agent

# Start the FastAPI server
python server.py

# In a separate terminal, expose via ngrok
ngrok http 8000

5. Twilio Setup

• Point your Twilio Phone Number's Voice Webhook to: https://<your-ngrok-url>/twilio/answer
• Set the Status Callback to: https://<your-ngrok-url>/twilio/status

---

🏗️ Architecture

For a deep dive into the system design, data flow, and memory management, see ARCHITECTURE.md.

---

🧠 Memory & Ground Truth

• ground_truth_context.txt: Edit this file to give the AI its identity, schedule, and authoritative knowledge.
• conversation_store/: Contains full transcripts and semantic indices of past calls.
• profile_memory.json: Where the AI stores facts it learns about you and other callers.

---

📊 Performance expectations (CPU)

Stage	Latency
Transcription (Whisper)	~600ms
Brain (LLM First Token)	~800ms
Voice (TTS Generation)	~150ms
Total Response Time	~1.5s - 2.5s

---

📄 License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).

You are free to use, study, modify, and redistribute this software under the terms of the AGPL-3.0.

If you distribute modified versions or deploy this software as a network service, you must also make the corresponding source code available under the same license.

See the LICENSE file for the complete license text.