Life & fun · November 26, 2025
Real-time system that tracks how news spreads across 200k websites
I built a system that monitors ~200,000 news RSS feeds in near real-time and clusters related articles to show how stories spread across the web. It uses Snowflake’s Arctic model for embeddings and HNSW for fast similarity search. Each “story cluster” shows who published first, how fast it propagated, and how the narrative evolved as more outlets picked it up. Would love feedback on the architecture, scaling approach, and any ways to make the clusters more accurate or useful. Live demo: https://yandori.io/news-flow/
In plain words
This system monitors approximately 200,000 news RSS feeds to track how stories spread across websites in near real-time. It clusters related articles using Snowflake's Arctic model for embeddings and HNSW for fast similarity search, then displays which outlets published first, how quickly stories propagated, and how narratives evolved as more publishers covered them. It serves journalists, researchers, and anyone interested in understanding news distribution patterns across the web.
written from the facts on this page · September 2026
Does a similar job
all alternatives →- SCSemantic clusters and embeddings for 500k Hacker News comments2024 · app.airtrain.ai · ▲114
- RBRSS Brain2022 · rssbrain.com · ▲139
I wrote an RSS reader called RSS Brain recently. The motivate is the current RSS readers either don't sort articles by priority, which makes it hard to read posts from HackerNews and Reddit, or sort it with some "smart" algorithm which I don't trust. I also like the Google News feature that can show related story from different source, but the sources are not configurable and the algorithm is not transparent either. So with RSS Brain, I implemented these features: * Recommend related articles from the feeds of your choice. It's backed by ElasticSearch and the algorithm is described on RSS…
- TATidalWaves API – live, tokenized news metadata from around the world2020 · tidalwaves.io · ▲124
- ISI Scraped Hacker News for TLD Popularity2023 · v01.io · ▲68
- NPNews Pulse – Real-time global news feed, 475 sources, no algorithmFeb 2026 · news-alert-eta.vercel.app · ▲9
Former investigative reporter turned developer. I built a simple breaking-news monitor because tracking events across platforms is a mess now that Twitter’s unreliable. Bluesky is the backbone (a fraction of twitter, but still lots of journalists and OSINT folks), plus RSS, Telegram, Reddit, YouTube and Mastodon. Everything is one chronological feed with no algorithm, clear source labels, and lightweight activity detection when a region spikes above baseline (frequency math, not LLMs). But we do have a (hopefully non-obtrusive) AI-generated recent/post summary. Been building for a…
More life & fun this month
the category →- TL
Life & fun · 11d ago · louisabraham.github.io
Photosynthesis fires two of your iPhone
Life & fun · 29d ago · photosynthesis.camera
- CCCreatium Coach▲319
Your multimedia mentor that takes you from mid to great
Life & fun · 11d ago · producthunt.creatium.info
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 29d ago · solouno.io
Launched alongside, November 2025
the whole month →
- IB
Life & fun · Nov 2025 · bitsnpieces.dev
- BBoing▲782
Life & fun · Nov 2025 · boing.greg.technology