feat: 5 feed quality improvements (no Reddit, source cap, junk filter, Google News sources, niche priority)#1
Open
fainir wants to merge 1 commit into
Open
feat: 5 feed quality improvements (no Reddit, source cap, junk filter, Google News sources, niche priority)#1fainir wants to merge 1 commit into
fainir wants to merge 1 commit into
Conversation
…nk filter, Google News sources, niche priority - #1 Remove all Reddit RSS from GLOBAL_SOURCES; replace with quality journalism (Wired, MIT Tech Review, The Decoder, VentureBeat, Schneier, InsideClimateNews, CleanTechnica, KDnuggets, thenewstack.io, etc.) - #2 Add capBySource() in classify.ts — max 20% of any single domain per AI batch - #3 Pre-filter job postings in isJunkArticle (we're hiring, open position, looking for developer, etc.) - #4 Fix Google News source names — use per-item <source> publisher tag via rssParser customFields, fall back to article URL domain - #8 Niche feed priority in scanGoogleNews — query recent item counts, feeds with 0 recent items go first Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fainir
added a commit
that referenced
this pull request
May 15, 2026
Four perf wins that compound. From "Enter on URL" to "first article visible", combined effect is roughly: cold cache, 1st visit: ~1200ms → ~250ms warm cache, revisit: ~800ms → ~50ms #1 + #5 — Server-side initial article fetch (RSC): /[feed]/page.tsx is now a server component that fetches the feed payload during the SSR pass and passes it to the existing client component as initialItems. The HTML response contains the first 50 articles inline — no client-side round-trip is needed on first paint. The client takes over on hydration for reactions / customize / pagination / theme. In-app navigation between feeds falls back to the existing localStorage SWR path (initialSlug !== current slug). #2 — Cron-driven cache pre-warming: After every scan-and-match run that actually classified new items, warmFeedCache() hits /api/public/feeds for every system feed and the top 20 most-followed community feeds. The L2 (Redis) cache is always warm so the first real visitor in the 60s TTL window gets HIT-L2 instead of paying the Supabase round-trip. #3 — preconnect / dns-prefetch to image CDNs: TLS handshakes to cdn-images-1.medium.com and the long-tail favicon hosts now happen in parallel with HTML parsing, instead of starting only after the first <img> tag fires. #4 — Tab prefetch on hover / touch / focus: New useFeedPrefetch hook fires fetch() to the public feed API the moment the user hovers, taps (touchstart), or keyboard-focuses a tab. Cache-Control on the response means the browser keeps it in memory for the actual navigation that follows. Deduped per slug so a hover-storm doesn't fire 14 requests. Edge runtime (#6) deferred — would require migrating from Railway Redis to Upstash for HTTP-based access. Given L1 already serves in ~5ms, the edge-routing win is marginal vs the migration cost. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fainir
added a commit
that referenced
this pull request
May 15, 2026
#1 Cache warming - scan-and-match cron now calls warmFeedCache() unconditionally on every tick, not only when new articles were classified. The old gate caused MISS on system-feed endpoints when classification produced no new inserts — verified live in last QA pass: AI, Tech, Startups, Dev all showed X-Cache: MISS. - Bumped server-side L1 + L2 TTL from 60s → 5min (RESPONSE_TTL_MS in feeds + TTL_MS in feed-by-slug, plus DEFAULT_TTL_MS in cache.ts). Cache now survives between the 15min cron ticks instead of expiring after 60s. Browser/CDN Cache-Control stays at 60s so revisits still revalidate frequently. #3 Dashboard subtree reachable - Added "Settings" menu entry in both the homepage and /[feed] signed-in 3-dot menus, linking to /dashboard/settings. From there the existing dashboard pages (account, digest, notifications, etc.) are reachable via in-page navigation, so they're no longer orphans. Kept the routes themselves (they have real, useful UI — 200–600 lines each — for feed-schedule preferences, push notifications, feed import/export). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
capBySource()inclassify.tslimits any single domain to max 20% of articles per AI batch, preventing a single noisy publisher from monopolizing classification.isJunkArticle()now drops job announcements ("we're hiring", "open position", "looking for a developer") before they reach the pool.rss-parserwithcustomFields: { item: [["source", "gnSource"]] }so Google News items show the actual publisher (e.g. "Wired") instead of the query string. Falls back to article URL domain.scanGoogleNewsnow queries recent item counts per feed. Feeds with 0 items in the last 24h (niche/underserved) are processed first.Test plan
article_poolfor absence of reddit.com domain articles from global scanfeed_itemsfor no job posting titles🤖 Generated with Claude Code