nowfound

Dev tools · February 24, 2012

I wrote a tiny Python-based HN crawler with scrapy

A scrapy-based Hacker News crawler. Contribute to mvanveen/hncrawl development by creating an account on GitHub.

I wrote a tiny Python-based HN crawler with scrapy launched on February 24, 2012 with 53 votes, #35 of 360 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 14 years on.

Visit github.com ↗Alternativestop 10% of February 2012

In plain words

HNCrawl is a Python-based crawler built with Scrapy that extracts data from Hacker News. It is designed for developers who need to programmatically collect and process Hacker News content. The tool is open source and available on GitHub for community contributions and development.

written from the facts on this page · September 2026

Does a similar job

all alternatives →
  • AD
    A discovery-focused search engine for Hacker News2024 · hn.trieve.ai · ▲193

    We (Nick, Dens, Denzell, Fede, Drew, Aaryan, and Daniel) have been building HN Discovery, a discovery-focused search engine for Hacker News, in our spare time for the past 6 months and are excited to show it! It adds the following features relative to the existing keyword search interface and preserves the existing ones: - no-JS version (hnnojs.trieve.ai) - site:{required_site} and site:{negated-site} filters - public analytics - LLM generated query suggestions based on random stories - recommendations - dense vector semantic search - SPLADE fulltext search - RAG AI chat - order by…

  • WH
    Wayback HN, a cross between the Wayback Machine and Hacker News2015 · waybackhn.com · ▲503
  • Crawly2016 · crawly.diffbot.com · ▲606

    Never write another web scraper

  • My recommendation engine for Hacker News2023 · hn-recommend.julienc.me · ▲314

    Hi! I’m Julien and I built a recommendation engine for Hacker News. I feel like this website is a gold mine. Every day, I find some very interesting stories about a topic. And sometimes, I want to find other stories covering that same topic but I can’t. Hacker News has years of history of awesome discussion and ressources. Unfortunately, I think HN Algolia isn’t helpful in searching these old threads. As a student, I want to learn a lot from this website. This is why I created HN Recommend. Input a sentence or the URL of an article, and get the most popular and similar posts from Hacker…

  • I made a search engine for Hacker News2024 · hackernews.demo.vectara.com · ▲211

    I love HN but always felt the search with algolia is okay but does have some limitations. Since I work at Vectara I decided to try and create a better search for HN. It's based on data from roughly the last 6 months of HN stories and comments. Would love to hear feedback and how useful this is relative to the existing search.

  • Hackerhunt – categorised curation of Show HN submissions2017 · hackerhunt.co · ▲335

More dev tools this month

the category →
  • /dev/null▲468

    a social network for devs

    Dev tools · 2d ago · dev-null.sh

  • Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

    open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

    Dev tools · 27d ago · github.com

  • Reladraw – A diagram language where you decide where to place things▲411

    Contribute to reladraw/reladraw development by creating an account on GitHub.

    Dev tools · 25d ago · github.com

  • OpenBot▲362

    Grok Bot alternative: free, local, open-source, multiplayer

    Dev tools · 22d ago · openbot.run

  • Talanoa▲340

    Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

    Dev tools · 20d ago · talanoa.email

  • CoreSpeed▲332

    One MCP for everything your agents need: apps, memory, tools

    Dev tools · 27d ago · corespeed.io

Source: Show HN post ↗

Launched alongside, February 2012

the whole month →