Alternatives
Products that do what Cerebro: a librarian for the 463-exabyte-a-day internet does
You missed a miracle while you slept. Between midnight and breakfast people uploaded five hundred hours of video every single minute. That is seven hundred and twenty thousand fresh hours in one day, enough footage to keep you watching until 2107 if you pressed play right now. And that is only YouTube. IDC says humanity will push out about 463 exabytes of new data every day in 2025. Stack that on hard drives and the column reaches the Moon and comes halfway back. Insane right? it get's worst. McKinsey finds that knowledge workers already burn a full work-day each week just hunting for…
- 1HN
Mar 2026 · huggingface.co
- 22G
Community, All the HN belong to you. This is an archive of hacker news that fits in your browser. When I made HN Made of Primes I realized I could probably do this offline sqlite/wasm thing with the whole GBs of archive. The whole dataset. So I tried it, and this is it. Have Hacker News on your device. Go to this repo (https://github.com/DOSAYGO-STUDIO/HackerBook): you can download it. Big Query -> ETL -> npx serve docs - that's it. 20 years of HN arguments and beauty, can be yours forever. So they'll never die. Ever. It's the unkillable static archive of HN and it's…
Dec 2025 · hackerbook.dosaygo.com
- 3SH
Reddit's API is effectively dead for archival. Third-party apps are gone. Reddit has threatened to cut off access to the Pushshift dataset multiple times. But 3.28TB of Reddit history exists as a torrent right now, and I built a tool to turn it into something you can browse on your own hardware. The key point: This doesn't touch Reddit's servers. Ever. Download the Pushshift dataset, run my tool locally, get a fully browsable archive. Works on an air-gapped machine. Works on a Raspberry Pi serving your LAN. Works on a USB drive you hand to someone. What it does: Takes compressed data dumps…
Jan 2026 · github.com
- 4
- 5OA
I built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files. Instead of training the model to generalize, I train a 900KB transformer to memorize a single file and predict the next byte. Those predictions are fed into an arithmetic coder to produce the compressed output. On a 100MB NYC taxi CSV, it compresses to about 7MB (~0.5 bits/byte). On a 100MB slice of enwik9, it compresses to about 21MB (~1.68 bits/byte). It's pretty slow right now (roughly 20–30 minutes of training and 45 minutes each for compression and decompression on my…
Jun 2026
- 6RT
I built a system that monitors ~200,000 news RSS feeds in near real-time and clusters related articles to show how stories spread across the web. It uses Snowflake’s Arctic model for embeddings and HNSW for fast similarity search. Each “story cluster” shows who published first, how fast it propagated, and how the narrative evolved as more outlets picked it up. Would love feedback on the architecture, scaling approach, and any ways to make the clusters more accurate or useful. Live demo: https://yandori.io/news-flow/
Nov 2025 · yandori.io
- 7
- 8PS
2017 · github.com
- 9DT
Magnet link: magnet:?xt=urn:btih:44c65b5779d9d8021e002584fa73740f36d052a6&dn=10m_hn_comments_sorted Go to https://hn-archive.appspot.com/ for the torrent file / source code. I'll be semi-frequently checking the story and answering any questions which may come up.
2015
- 10
- 11IM
I scraped every Show HN from the last 12 months (41,301 posts) plus the full comment tree of every launch with 10+ comments, ~100k comment timestamps, all from the Algolia HN API. The median launch gets 2 points and 0 comments. For launches that do get traction, half the comments they'll ever get arrive within 7.2 hours and 90% within 26, and the top decile decays on the same clock as everyone else. Vote timestamps aren't public, so comment timing is the attention proxy; caveats are in the post. Everything reproduces from the repo with one command…
Jul 2026 · jonno.nz
- 12LT
I struggle with wanting to read heaps of stuff I come across on the internet daily, as well as being subscribed to a bunch of tech-related newsletters that I never get around to reading. I made this to get through that content much faster while working or otherwise multitasking. Check it out and please lmk your thoughts! Also, curious if others also feel this pain of too much interesting content to get through daily. FWIW I probably listen to this for >1 hour daily atm.
2024 · lavalamp.ai
- 13II
Hi HN, I'm Roland, and for the past few weeks, I've been building AllBSides — a directory of every BSides conference talk uploaded to YouTube. As of today, 8,643 talks from 5,927 speakers across 227 chapters in 68 countries. Combined runtime is 280 days. The transcripts come to about 60 million words. The archive came together in stages: 1. Manually map every BSides chapter's YouTube channel 2. Pull every video and transcript from Supabase 3. Run each transcript through Haiku for tag extraction (tools, topics, difficulty, team, talk style, research method, and much more) 4. Run results…
May 2026 · allbsides.com
- 14UT
4 months ago, our announcement for our free, multi-node, petabyte-scale time-series database generated 200+ comments: https://news.ycombinator.com/item?id=23272992 Many commented on the Timescale License, our source-available license governing advanced features (eg multi-node). Most were positive, but some pointed out drawbacks (who says HN can't be productive!). That spurred an internal debate. Today we're happy to share an updated Timescale License with new rights, including the "right-to-repair" and "right-to-improve." And we’re doubling-down on our community, making all…
2020
- 155D
597 days ago, I posted about the project our team was developing - https://news.ycombinator.com/item?id=3183322 - here on HN, and today we're finally taking the wraps off what we've been working on since then. At it's core, what we've been building is an end-to-end solution for creating, storing, and working with structured content in a way that's accessible to anyone. You can check out an introduction - http://team.marquee.by/introducing-marquee/ and dig into the technical/philosophical details of our approach -…
2013
- 16BE
I built this because I am perpetually drowning in file transfer hell. As a systems engineer, I am constantly moving massive files. Whether it's getting new releases of custom ISOs to clients, downloading entire filesystems to spin up clones, or uploading massive support packages to L3. I always have to get something to somebody. Usually, it's something large that lives on a headless server. I can't tell you how many times I've modified my 'clone' script to tar most of a filesystem over ssh ("hello --exclude, my old friend...") just to get 10GB that someone will need for two days. Tedious is…
Jan 2026 · bucketlabs.org
- 17
Find out if the internet would destroy it in 14 seconds.
Jun 2026 · vnta.agency
- 18CW
Hello HN, Lucas here. I’ve been working with BigQuery for ~5 years, mostly in large (petabyte-scale) environments. Over time we ended up spending a lot of money and engineering effort just trying to understand where costs were coming from, why and how to optimize them. At some point we decided to stop, leverage all our past experience and spend a full cycle building tooling focused on cost visibility and optimization. The main goal was to regain ownership of cost data and make it possible to understand our cost structure in under a minute, while aligning the views of engineering and FinOps…
Jan 2026 · cloudclerk.ai
- 19A1
Hello HN, Wish you a very Happy New Year. StatPecker is our baby project. A cute little infographics generation tool for anybody seeking insights. We recently dropped a new feature, to help user upload CSV upto 50mb or in other words data with ~1 million rows. We wanted to make sure user data doesn’t get leaked, so we developed a mechanism where the CSV is analysed on user’s device. We only use our AI APIs to generate SQL queries on the fly and query the DB on user’s machine, along with the final response for aggregated data. The result turns out to be quite fast. Would love you to try it out.
Jan 2026 · app.statpecker.com
- 20ID
Hi HN! I put together a 4-hour conversational audiobook tracing their history—from IBM punch cards to the internet, the cloud, hyperscalers, and today’s AI factories. It’s part of a new long-form series on the history of infrastructure (first two episodes were on coal). Would love your thoughts—both on the story and on whether this “conversational audiobook” format works.
Sep 2025 · stepchange.show
- 21IM
Hi there! I've been working with data in one form or another, professionally, for about 5 years. I've been thinking about my own personal data and how it's used for at least twice that long. I've been sort of building something in my head for a while that solves my own problem and, in the beginning of this year, I found the opportunity to spend some time building it out. I'll leave the detailed explanation to the blog post but, in short, I built what amounts to an API crawler combined with a data processor to help you download your personal data from 3rd party services and work with it using…
2024 · joshcanhelp.com
- 22NP
Former investigative reporter turned developer. I built a simple breaking-news monitor because tracking events across platforms is a mess now that Twitter’s unreliable. Bluesky is the backbone (a fraction of twitter, but still lots of journalists and OSINT folks), plus RSS, Telegram, Reddit, YouTube and Mastodon. Everything is one chronological feed with no algorithm, clear source labels, and lightweight activity detection when a region spikes above baseline (frequency math, not LLMs). But we do have a (hopefully non-obtrusive) AI-generated recent/post summary. Been building for a…
Feb 2026 · news-alert-eta.vercel.app
- 23WS
Hey HN!! Many of us enjoyed the "If YouTube had actual channels" Show HN post last week. As developers of a similar project, we wanted to share our approach to building a simulated live experience. We’ve spent around 18 months working on this and have managed to add some advanced features, including content-based interactivity. We are super excited to be releasing both the live stream and the code behind it. Our idea was to combine the communal watching experience of television with the modern trappings of the interactive web. If you go to the stream, you will be watching along with everyone…
2024 · github.com
- 24CA
Hi HN! I started working in Cancer Research at the University of Hawaii Cancer Center 4.5 years ago in March 2018. If you thought programming and computers were complex, WAIT until you start trying to wrap your head around cancer. We're talking Orders of Magnitude more complexity. I know it might seem outrageous that a lowly CSV file will help in the quest to build a cure, but I truly believe it might. I think what seems like miracles always comes down to grunt data collection, with a few moments of key insights. Anyway, the site will launch November 1st. Please give it a star and a share…
2022 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →