GAIA Agent

A tool-using AI agent built for the Hugging Face Agents Course — Unit 4 Final Assignment. It answers questions from the GAIA benchmark (Level 1, validation subset), submits its answers to the course's scoring API, and reports the resulting score — all from a one-click Gradio UI.

What is GAIA?

GAIA ("General AI Assistants") is a benchmark of real-world questions that require multi-step reasoning, tool use, and often an attached file (image, audio, spreadsheet, code, or a YouTube link) to answer correctly. For this assignment, the agent is evaluated against 20 filtered Level-1 questions, graded by exact string match.

How it works

1. The Gradio app (app.py) authenticates the user via Hugging Face OAuth.
2. It fetches the 20 evaluation questions from the course's scoring API (GET /questions).
3. For each question, GAIAAgent (agent.py) — a smolagents CodeAgent running on GPT-4o — reasons through the task, calling tools as needed.
4. Every answer is passed through a cleanup step that strips extra formatting so it matches the exact-match grader.
5. All answers are submitted in one batch (POST /submit), and the resulting score is displayed alongside a table of each question and the agent's response.

Agent tools

Tool	Purpose
WebSearchTool	General web search
wikipedia_content	Fetches Wikipedia article text (and specific sections) directly via the MediaWiki API — used instead of generic scraping for reliable table/list extraction
visit_webpage	Reads and cleans text from any other webpage
download_task_file	Downloads a question's attached file, with a fallback to the GAIA dataset on the Hub if the scoring API's file endpoint fails
read_task_file	Reads a downloaded file's text content
run_python_file	Executes an attached .py file in a sandboxed subprocess with a timeout, falling back to static code inference if execution is too slow
transcribe_audio	Transcribes attached audio using Whisper
analyze_image	Answers questions about an attached image using a vision model
get_youtube_transcript	Pulls captions/transcript text from a YouTube link

The agent also has authorized access to pandas, numpy, openpyxl, and PIL for inline data and image analysis (e.g. reading .xlsx attachments).

Project structure

.
├── agent.py           # GAIAAgent: tool definitions, answer cleanup, OpenAI/smolagents setup
├── app.py              # Gradio UI: HF login, question fetch, run + submit loop
├── requirements.txt    # Python dependencies
└── README.md

Setup

1. Clone the repo

git clone https://github.com/koushiikk/GAIA_Agent.git
cd GAIA_Agent

2. Install dependencies

pip install -r requirements.txt

3. Set environment variables

Variable	Required	Description
OPENAI_API_KEY	Yes	Used for the agent's LLM calls, vision analysis, and Whisper transcription
OPENAI_MODEL	No	Overrides the default model (gpt-4o)
HF_TOKEN	No	Needed only as a fallback to download gated GAIA attachments directly from the Hub if the scoring API's file endpoint is unavailable

4. Run locally

python app.py

This launches a local Gradio interface. Log in with your Hugging Face account, then click Run Evaluation & Submit All Answers.

Deploying as a Hugging Face Space

This project is set up to run as a Gradio Space out of the box (sdk: gradio, app_file: app.py). Push it to a Space, add OPENAI_API_KEY (and optionally HF_TOKEN) as Space secrets, and keep the Space public — the scoring API records your Space's code URL as proof of submission for the leaderboard.

Design notes

• Fails fast on bad API keys/quota — the agent pings OpenAI on startup so a bad key or exhausted quota surfaces immediately instead of hanging through retries.
• Answer cleanup (clean_answer) — strips code fences, "FINAL ANSWER:"-style prefixes, and quote wrapping; converts number-word answers to digits for "how many" questions; and trims to a first name where the question asks for one — all aimed at surviving exact-match grading.
• Bounded reasoning — each question is capped at 8 agent steps to keep runs fast and predictable.
• Resilient file execution — .py attachments run with a 10-second timeout; if that's too slow, the agent falls back to statically reading the code and inferring its printed output.

Reference

• Unit 4: The Final Hands-On — Hugging Face Agents Course
• GAIA Benchmark Leaderboard