Client Scraper — Free Lead Finder for Web-Design Cold Calling

Finds businesses on Google Maps (niche + location) that are popular but have no website, enriches top leads with their Instagram/Facebook presence, and exports a formatted Excel file ready for cold calling. 100% free: no API keys, no paid services — Playwright drives the browser, SQLite dedupes locally.

Setup

python -m venv .venv
.venv\Scripts\python -m pip install -r requirements.txt
.venv\Scripts\python -m playwright install chromium

Usage

# single niche + location
.venv\Scripts\python main.py --niche "restaurants" --location "Lucknow"

# tighter filters (businesses with 50+ reviews, rated 4.5+)
.venv\Scripts\python main.py --niche "salons" --location "Kanpur" --min-reviews 50 --min-rating 4.5

# batch file (niche,location per line, header optional)
.venv\Scripts\python main.py --batch batch.csv

# watch it run / solve a CAPTCHA manually if Google shows one
.venv\Scripts\python main.py --niche "gyms" --location "Delhi" --headless=false --slow-mode

# re-scrape everything, ignore cache
.venv\Scripts\python main.py --niche "gyms" --location "Delhi" --force-refresh

# skip social enrichment (faster, fewer requests)
.venv\Scripts\python main.py --niche "gyms" --location "Delhi" --skip-social

Flags

Flag	Default	Meaning
--niche / --location	—	Category and area (e.g. "cafes", "Alambagh")
--batch FILE	—	CSV of niche,location combos
--min-reviews N	20	Drop businesses with fewer reviews
--min-rating X	4.0	Drop businesses rated below X
--max-results N	100	Hard cap per search (anti-block)
--output PATH	auto	Output .xlsx (single combo only)
--slow-mode	off	2.5x longer delays between actions
--headless=false	true	Show the browser (watch / solve CAPTCHAs)
--force-refresh	off	Re-scrape and export even if cached
--skip-social	off	Skip social enrichment
--social-limit N	30	Max leads enriched per run (top-scored first)
--examples	—	Print copy-paste example commands and exit

How it works

0. Live progress UI — every phase shows a spinner, progress bar, elapsed time and an ETA (per-listing during scraping, per-lead during social enrichment, per-combo across batch runs). Stdlib-only, no extra deps.

1. Scrape — Playwright opens google.com/maps/search/{niche} in {location}, scrolls the results feed until it plateaus, then visits each place page and reads name, address, phone, rating, review count, category, website, status, and hours from the detail panel. Random 2–5s delays between every action.

2. Filter — drops businesses with a real website (a link to Instagram/Facebook/Linktree does not disqualify — flagged Social-link-only, those are prime leads), rating below min, too few reviews, or permanently closed.

3. Score — 0–100: review count (weight 45), rating (35), open status (20), IG followers bonus (5). Sorted descending.

4. Enrich (top 30 by score) — DuckDuckGo search for "{name} {location} instagram", grabs the first plausible profile URL, then reads follower count from the public page's meta tags. Login walls are skipped gracefully — URL kept, count blank.

5. Export — output/leads_{niche}_{location}.xlsx:

   • Leads sheet: all 17 columns, Excel Table with filters, frozen header, green/yellow/red row coloring by score, Call Status dropdown (Not Called/Called/Interested/Not Interested/Follow Up), blank Notes.
   • Summary sheet: totals, leads per category, average score.
   • Failures sheet: any listing that errored, so nothing silently disappears.

6. Cache — SQLite leads.db dedupes by name|address hash; already-seen businesses are skipped on later runs (use --force-refresh to override). Every run is logged with timestamps and counts.

Staying unblocked (read this)

• Don't run multiple times in a row. One session at a time, delays built in, hard cap of 100 listings/search.
• If Google shows a CAPTCHA, the tool stops with a message — wait 15–60 minutes before retrying. Hammering it gets your IP blocked.
• Use --slow-mode and small --max-results for new locations you've never scraped.
• Prefer batch mode for many combos but spread runs across days.

Troubleshooting

• "Could not find the Google Maps results feed" — Google changed the page layout (selectors are in maps_scraper.py). Set LEAD_DEBUG=1 to dump panel text, then update the fallback selectors.
• Everything filtered out — your niche/location genuinely has few results, or --min-reviews/--min-rating are too strict. Loosen them and retry.
• No Instagram followers — normal: profile is private or requires login. The URL is still saved; use it when cold-calling.

Ethics

Public data only, no logins, no bypassing anything, per-listing error handling, respectful rate limits. Only call businesses whose info you found on public pages.

Contributors

• stunninghacker — creator
• pranjalmauryya — contributor