Concept search engine based on most popular HN submissions
Thesis: Sites that do well on hacker news will tend to be sites with high quality content. Tools: Hacker News Big Query, python, Google CSE Steps: 1. Using HN Big Query, get all unique domains with more than 3 stories with more than 50 points (query link [1]). Sort by percentage of such stories to total number of stories. By doing that, at the top you will get sites like blog.geoffralston.com that have 3 out of 3 submitted stories get more than 50 points (100% !). Or lucumr.pocoo.org had 46 out of 124 total stories reach 50+ points! Talking about good writing. We cut the list at 2,500 sites…
What it does
In the maker’s words, at launch
Thesis: Sites that do well on hacker news will tend to be sites with high quality content. Tools: Hacker News Big Query, python, Google CSE Steps: 1. Using HN Big Query, get all unique domains with more than 3 stories with more than 50 points (query link [1]). Sort by percentage of such stories to total number of stories. By doing that, at the top you will get sites like blog.geoffralston.com that have 3 out of 3 submitted stories get more than 50 points (100% !). Or lucumr.pocoo.org had 46 out of 124 total stories reach 50+ points! Talking about good writing. We cut the list at 2,500 sites , where the popular to submitted ratio is still at enviable 12%. 2. Add to this list all sites that had exactly one submission and that only submission ever from that domain had 300+ points on HN. I call them unexplored one hit wonders and thesis is that there are probably other gems on the domain just not ssubmitted yet. [2] 3. Now we have about 3,000 sites total. We will use Google CSE engine which allows up to 2,000 sites through annotations [3]. We have to clean the data now. - Check if the domain still resolves. Sadly about 400 these high quality sites do not anymore. - Check for redirects For example this site is no longer on its old address: https://david.weebly.com ... 302 Found (0.153) http://www.david.blog/ ... 200 OK (0.0676) - Check for all other sorts of weird errors This took most of the day :) I used modified version of [4] 4. Manually clean the list from news sites that made it on (nytimes, usatoday...) 5. If you want to check if your site is on the list, check [5]. If you are on the list congrats! 5. Finally, here is the end result: https://cse.google.com/cse?cx=014479775183020491825:c2lrlzrogb5 Search cream of the crop of HN submitted sites! Let me know if you find this useful!
Does the same job
all alternatives →- MRMy recommendation engine for Hacker News2023 · hn-recommend.julienc.me · ▲314
Hi! I’m Julien and I built a recommendation engine for Hacker News. I feel like this website is a gold mine. Every day, I find some very interesting stories about a topic. And sometimes, I want to find other stories covering that same topic but I can’t. Hacker News has years of history of awesome discussion and ressources. Unfortunately, I think HN Algolia isn’t helpful in searching these old threads. As a student, I want to learn a lot from this website. This is why I created HN Recommend. Input a sentence or the URL of an article, and get the most popular and similar posts from Hacker…
- IMI made a search engine for Hacker News2024 · hackernews.demo.vectara.com · ▲211
I love HN but always felt the search with algolia is okay but does have some limitations. Since I work at Vectara I decided to try and create a better search for HN. It's based on data from roughly the last 6 months of HN stories and comments. Would love to hear feedback and how useful this is relative to the existing search.
- 2G22 GB of Hacker News in SQLiteDec 2025 · hackerbook.dosaygo.com · ▲730
Community, All the HN belong to you. This is an archive of hacker news that fits in your browser. When I made HN Made of Primes I realized I could probably do this offline sqlite/wasm thing with the whole GBs of archive. The whole dataset. So I tried it, and this is it. Have Hacker News on your device. Go to this repo (https://github.com/DOSAYGO-STUDIO/HackerBook): you can download it. Big Query -> ETL -> npx serve docs - that's it. 20 years of HN arguments and beauty, can be yours forever. So they'll never die. Ever. It's the unkillable static archive of HN and it's…
- HSHacker Search – A semantic search engine for Hacker News2024 · hackersearch.net · ▲233
Hi HN! I'm Jonathan and I built Hacker Search (https://hackersearch.net), a semantic search engine for Hacker News. Type a keyword or a description of what you're interested in, and you'll get top links from HN surfaced to you along with brief summaries. Unlike HN's otherwise very valuable search feature, Hacker Search doesn't require you to get your keywords exactly right. That's achieved by leveraging OpenAI's latest embedding models alongside more traditional indexes extracted from the scraped and cleaned up contents of the links. I think there are many more interesting things…
- ATAnalyzing top HN posts with language models2022 · ▲117
Hi HN, I spent a few weeks looking at the top HN posts of all time. This included exploration, clustering, creating visualizations, and zooming in on what (to me personally) seems like some of the best discussions on here. Three things in this post: 1- The interesting groups of HN posts 2- The interactive visualizations that you can explore in your browser 3- The data from this exploration -- this includes CSV of the titles as well as the text embeddings of 3,000 Ask HN articles. Blog post about this whole process here: [1] ============ 1- The interesting groups of HN posts From the…
- ADA discovery-focused search engine for Hacker News2024 · hn.trieve.ai · ▲193
We (Nick, Dens, Denzell, Fede, Drew, Aaryan, and Daniel) have been building HN Discovery, a discovery-focused search engine for Hacker News, in our spare time for the past 6 months and are excited to show it! It adds the following features relative to the existing keyword search interface and preserves the existing ones: - no-JS version (hnnojs.trieve.ai) - site:{required_site} and site:{negated-site} filters - public analytics - LLM generated query suggestions based on random stories - recommendations - dense vector semantic search - SPLADE fulltext search - RAG AI chat - order by…
More dev tools this month
the category →
The first open-source price index for GPU compute Discussion | Link
Dev tools · 10d ago · producthunt.com
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Launched alongside, October 2019
the whole month →- BBel▲1,288
Life & fun · 2019 · paulgraham.com
Create free personalized illustrations from 3000+ elements
Growth · 2019 · icons8.com
