An open source web crawler for the Mwmbl non-profit search engine
A browser extension that can be installed by volunteers to participate in mwmbl distributed crawling. - mwmbl/crawler-extension
An open source web crawler for the Mwmbl non-profit search engine launched on June 20, 2022 with 6 votes, #1,147 of 3,005 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 4 years on.
Does a similar job
all alternatives →
I'm building a non-profit search engine2021 · github.com · ▲441An open source, non-profit web search engine. Contribute to mwmbl/mwmbl development by creating an account on GitHub.
Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116Distributed web crawler admin platform for spiders management regardless of languages and frameworks. - tikazyq/crawlab
- Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282
Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…
Headless Chrome Crawler2018 · github.com · ▲172Distributed crawler powered by Headless Chrome. Contribute to yujiosaka/headless-chrome-crawler development by creating an account on GitHub.
More dev tools this month
the category →
Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.
Dev tools · 1d ago · github.com
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 29d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 27d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 24d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 22d ago · talanoa.email
Source: Show HN post ↗
Launched alongside, June 2022
the whole month →- AF
I've seen some interests in (simulation) video games here on HN so I thought I'd share a short version of our story. More than 6 years ago, me and my friend from university were playing around with an idea of making a game we always wanted to play. We worked on it on weekends but the progress was quite slow, especially due to so many dead ends and wasted effort. Eventually however, we solidified our direction and decided to take the risk to resign from our well paid SWE jobs and work on it full time. It took more than a year but yesterday we have finally released it on Steam:…
Life & fun · 2022
Life & fun · 2022 · benoitessiambre.com
min(DALL·E) is a fast, minimal port of DALL·E Mini to PyTorch - kuprel/min-dalle
Life & fun · 2022 · github.com
MakerBox▲621600 tools with generous free plans for indie entrepreneurs
Dev tools · 2022 · makerbox.club
Typedream CMS▲593Display your Notion data beautifully on your site, no-code
Dev tools · 2022 · typedream.com