fix: replace EasyOCR subprocess with persistent microservice (eliminates 30-60s cold start)

[tanvishdesai]

Problem

Every OCR scan submission spawned a fresh Python process that reloaded PyTorch + EasyOCR from scratch. Cold-start cost: 30–60 seconds per request. The timeout: 60000 in evaluation.ts existed specifically for this. Two simultaneous uploads = two full model loads competing for ~1.5GB RAM each → OOM risk on a 2GB droplet.

Solution

Replace the per-request subprocess with a persistent FastAPI microservice (ai-services/ocr_server.py) that loads EasyOCR once at startup via uvicorn's lifespan hook. Every subsequent request hits a warm reader.

	Before (subprocess)	After (microservice)
First request latency	30–60s (cold start)	~500ms (model already loaded)
Every request latency	30–60s (subprocess every time)	~500ms
2 concurrent uploads	~3GB RAM (OOM risk)	~800MB (shared model)
Timeout needed	60s	15s

Changes

• ai-services/ocr_server.py — new FastAPI server wrapping the existing easyocr_evaluator.run_fast_ocr() pipeline. No logic changes — same blue-pen filter, same component detection, same EasyOCR allowlist.
• backend/src/routes/evaluation.ts — both execFileSync OCR calls replaced with fetch() to http://127.0.0.1:8001/ocr. Timeout reduced from 60s → 15s.
• backend/src/config.ts — OCR_SERVICE_URL constant, overridable via env.
• ai-services/requirements.txt — add fastapi, uvicorn.
• ai-services/scripts/easyocr_evaluator.py — broaden paddle import guard from except ImportError to except Exception (covers binary-compatibility errors from the numpy ABI).
• fln-ocr.service — systemd unit for the OCR microservice (replaces ecosystem.config.js). Logs to /var/log/tenali/. Deploy via sudo cp fln-ocr.service /etc/systemd/system/ && sudo systemctl enable --now fln-ocr.
• ai-services/scripts/test_ocr_server.py — integration test that starts the server, waits for EasyOCR to warm, then verifies health + OCR response shape + sub-10s warm inference.

Startup dependency note: evaluation.ts now calls the OCR service at 127.0.0.1:8001 on every scan submission. The OCR service must be running before scan traffic hits the backend. If it is down, the backend returns a 502 error rather than crashing. On the droplet, ensure fln-ocr is enabled and started before fln-backend begins receiving traffic. The After=network.target in the unit file is sufficient for normal boot ordering; no hard After=fln-backend dependency is set because either service can restart independently.

Test results

[1] Health endpoint:               PASS  {'status': 'ok', 'easyocr_loaded': True}
[2] scan_1784808576635_file.jpeg:  PASS  elapsed=4.3s  tokens=['2', '3', '>', '>']
[3] scan_1784808718983_file.jpeg:  PASS  elapsed=3.5s  tokens=['2', '3', '>', '>']
[warm-repeat] Second call:         PASS  elapsed=3.52s

4/4 tests passed

Deployment (one-time on droplet)

pip install fastapi uvicorn
# Create log directory if it does not exist yet
sudo mkdir -p /var/log/tenali
sudo cp fln-ocr.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now fln-ocr

🤖 Generated with Claude Code

[jgupta05072003-code]

Hey @tanvishdesai , this looks good overall. The microservice approach and the performance improvements seem to solve the cold-start issue well.

I just have one deployment-related request before we merge this.

Our production server runs the other services using systemd, not PM2. We moved away from PM2 so that services automatically restart if they crash. Could you please replace ecosystem.config.js with a systemd service file for fln-ocr, following the same setup as our existing services?

Please use:

Type=simple
Restart=on-failure
RestartSec=3
WantedBy=multi-user.target
Logs should go to /var/log/tenali/

I can share one of our existing systemd unit files if you need a reference.

One more small thing. Please add a note in the PR description that evaluation.ts now connects to the OCR service at 127.0.0.1:8001. This means the OCR service should be running before the main backend starts receiving actual traffic.

This is not a blocker because, if the OCR service is down, the backend should return an error instead of crashing. However, it would be good to mention this for deployment and restart purposes.

Everything else, including the FastAPI implementation, tests, and configuration changes, looks good to me.

[tanvishdesai]

Thanks for pointing it out @jgupta05072003-code, have updated it. Replaced with using exactly the config you mentioned (, , , logs to ). Also deleted the PM2 file entirely since it's not used.

Added the startup dependency note in the PR description as well — basically just explains that if the OCR service is down, the backend returns a 502 instead of crashing, and that should be up before traffic hits .

Let me know if the service file needs any adjustments for your existing setup (like a different deploy user or path)!