Live demo · API docs · 90-second walkthrough
I built this to answer a specific question: can two different AI models, looking at the same photo of a dish, actually agree on what it costs, what's in it, and what to call it? Turns out — not always. That disagreement is the whole point of this project.
A restaurant owner takes one photo of a dish. Instead of typing out a menu entry by hand — name, description in three languages, price, allergens — the photo goes to two vision models at once: Gemini 2.5 Flash and an open-weight model on Groq (Qwen3.6-27B). Both are asked the exact same question, independently. If they agree closely, the dish gets auto-approved. If they don't, it gets flagged, and the owner sees both answers side by side before publishing.
I didn't want to build a wrapper around one model and call it done — the interesting engineering problem here was building a way to actually measure when two models disagree, and deciding what to do about it.
Upload a photo, get a result back in seconds:
Every dish stays editable before it goes live — the owner has the final say, not the AI:
The public menu supports English, Tamil, and Hindi — tap the language stamp to switch:
Each model's answer gets compared on three things, weighted differently because they don't matter equally:
- Name similarity (50%) — character-level comparison between what each model called the dish
- Allergen overlap (30%) — Jaccard similarity between the two allergen sets
- Price band overlap (20%) — how much the two suggested price ranges actually intersect
Below a 0.7 combined score, the dish gets flagged for manual review instead of auto-publishing. That threshold isn't arbitrary — it came out of testing against a small hand-labeled set of real dish photos (see below).
I labeled the correct name, price, and allergens for 8 real dish photos and ran both models against them.
| Gemini 2.5 Flash | Groq (Qwen3.6-27B) | |
|---|---|---|
| Exact name match | 88% | 50% |
| Avg. allergen recall | 100% | 100% |
| Avg. price-band overlap | 25% | 14% |
A few things stood out that I didn't expect going in:
- Both models overpriced almost everything, consistently, relative to what dishes actually cost at a casual restaurant here. I added a prompt hint about local pricing and it helped on 6 of 8 dishes — but had zero effect on naan specifically. My guess is naan shows up so often in international restaurant pricing data that one prompt hint couldn't outweigh it.
- Agreement went up after that fix, but accuracy barely moved. The two models started agreeing with each other more, while both were still wrong compared to my ground truth — which was a good reminder that two models agreeing with each other isn't the same as either of them being right.
- One photo — a plate of gulab jamun — got called "Rasmalai" by one model and "Gulab Jamun" by the other. Not a rounding error, an actual disagreement on what the dish was. That's exactly the kind of case this system is supposed to catch before it reaches a menu.
- The Groq model's Hindi output was noticeably rougher than its English or Tamil — sometimes outright ungrammatical. Since Gemini's answer is what gets saved when both models agree, this weakness mostly stays invisible to the final menu, but it's a real limitation worth naming.
8 photos is nowhere near enough to say one model is "better" — it's enough to prove the pipeline works and to find real, specific failure patterns worth writing down.
- AI: Gemini 2.5 Flash, Groq (Qwen3.6-27B), Pydantic for schema validation with retry-on-failure
- Backend: FastAPI, Supabase (Postgres with Row Level Security, Storage)
- Frontend: Next.js, TypeScript, Tailwind
- Testing/CI: pytest (37 tests), GitHub Actions
- Deployed on: Vercel (frontend), Render (backend, Docker)
# Backend
pip install -r requirements.txt --break-system-packages
cp .env.example .env # add your own keys — see below
uvicorn api.main:app --reload
# Frontend, separate terminal
cd frontend
npm install
npm run devFree API keys, no card required:
- Gemini: https://aistudio.google.com/apikey
- Groq: https://console.groq.com/keys
- Supabase: https://supabase.com
Tests don't need any live API keys — they check the schema validation and scoring logic directly:
pytest test_menuai_pipeline.py -vDocker, if you'd rather not set up a Python environment:
docker build -t menuai-api .
docker run -p 8000:8000 --env-file .env menuai-api- There's no real login yet — the restaurant ID is currently hardcoded on the frontend rather than tied to an authenticated owner. The backend uses Supabase's service-role key (which bypasses Row Level Security) and is responsible for its own authorization checks in code, as a deliberate stand-in until real auth exists. The one place RLS actually does the enforcing is the public menu endpoint, where it correctly restricts results to published dishes only.
- The 0.7 confidence threshold and the 50/30/20 weighting are starting points based on 8 test photos, not a properly calibrated setup — that would need a much bigger labeled dataset.
- Render's free tier spins down after 15 minutes of inactivity, so the first request after a while can take 30-50 seconds. Normal for a free-tier demo, not a bug.
- Groq's open-weight model lineup changes without much notice — if
qwen/qwen3.6-27bstops working, check https://console.groq.com/docs/vision for the current option.
- Real magic-link authentication, so the RLS gap above actually closes
- A price-correction layer trained on local pricing data, since prompt-level hints only got partway there
- A slightly bigger eval set — 50-100 photos would let me actually say something firm about which model performs better where, instead of just pointing at interesting individual cases