feat(suggestions): hybrid retrieval-grounded follow-ups behind feature flag

[yashgoel1331]

Summary

• Adds a hybrid suggestions pipeline that can ground follow-up questions in distilled search_documents retrieval evidence plus recent conversation context.
• Gates rollout behind SUGGESTIONS_HYBRID_ENABLED (default false) so production stays on conversation-only baseline until explicitly enabled.
• Adds observability for suggestions timing and input-mode selection without changing chat streaming behavior.

This PR contains exactly 2 commits cherry-picked onto current main:

1. 02f46a8 — suggestions scope guardrail (non-milk transaction limits)
2. 4ebd1d6 — hybrid suggestions pipeline + feature flag + docs

What changed

• Retrieval evidence extraction (app/services/chat.py): after each chat turn, distill current-turn search_documents tool returns into bounded snippets and cache as suggestions_shadow_{session_id}_{target_lang}.
• Hybrid input + quality gate (app/tasks/suggestions.py): when flag is enabled and retrieval passes deterministic checks, suggestions use last 3 conversation pairs + distilled evidence; otherwise fallback to last 5 pairs (conversation-only).
• Prompt guardrails (assets/prompts/suggestions_system.md): hybrid input usage rules, answerability constraints, and milk/cooperative scope limits.
• Feature flag (app/config.py, example.env): SUGGESTIONS_HYBRID_ENABLED.
• Instrumentation logs: suggestions_task_queued, chat_stream_complete, suggestions_shadow_evidence, suggestions_task_started, suggestions_input_mode, suggestions_cache_written.
• Docs: updated API_DOC.md and docs/CHAT_ENDPOINT_FE_INTEGRATION.md with current /api/suggest/ behavior and pipeline details.

Design notes

• Chat stream path is unchanged; suggestions run as FastAPI background tasks after streaming completes.
• Suggestions agent remains tool-free.
• Hybrid mode requires usable retrieval (min snippets/chars/overlap); weak/no retrieval automatically falls back.

Test plan

• With SUGGESTIONS_HYBRID_ENABLED=false, verify suggestions_input_mode ... mode=conversation_only retrieval_reason=hybrid_feature_flag_disabled.
• With SUGGESTIONS_HYBRID_ENABLED=true and a retrieval-heavy ag query, verify mode=hybrid retrieval_reason=ok.
• With hybrid enabled but no/weak retrieval, verify fallback to conversation_only with explicit retrieval_reason.
• Confirm chat streaming latency unchanged.
• Confirm /api/suggest/ returns JSON string array after generation.
• Spot-check suggestions avoid unsupported personal balance/passbook lookup prompts.

Made with Cursor