HyperCrawl

What it does

This is a zero-latency web crawler especially designed for retrieval-based LLM development

• 
  /agent by Firecrawl Dec 2025 · firecrawl.dev · ▲569

  While building mendable - we found that feeding LLMs well-structured markdown improved accuracy. We also found it surprisingly hard. We found some great tools online, but none reliably handled the entire process. We wanted an API that took a URL, crawled the pages in the URL, and gave us an easy-to-use, up-to-date markdown we could feed into our index. So, we released an open-source repo and an API that crawls and turns entire websites into a markdown with just a few lines of code The API handles: - Crawling without consistent sitemaps - Infra to handle running many crawling jobs - Proxying,…

• 
  Turn any website into a knowledge base for LLMs2024 · embedding.io · ▲305

  I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...

• 
  Webᵀ Crawl by Web Transpose2023 · ▲98

  Realtime web data for LLMs

• 
  The new Firecrawl /searchJul 2026 · docs.firecrawl.dev · ▲257

  Our most accurate Search API for AI agents.

• 
  Crawlee2022 · crawlee.dev · ▲229

  Build reliable web scrapers and robots, fast!

• 
  WebclawMay 2026 · webclaw.io · ▲17

  Turn any website into LLM-ready data

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 17d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 16d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 18d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 20d ago · stillwet.art

• 
  Onepin▲384

  AI voiceover with every name, price and date said right

  AI · 30d ago · onepin.ai

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 28d ago · world.pixverse.video

• 
  Voicenotes▲1,485

  AI note-taker that's truly intelligent

  AI · 2024 · voicenotes.com

• 
  Wegic▲1,179

  The first AI web designer & developer by your side

  Dev tools · 2024 · wegic.ai

• 
  Insighto▲1,108

  Ship features users want

  Growth · 2024 · insigh.to

• 
  Ivee▲1,087

  The B2B influencer marketing platform

  Growth · 2024

• 
  Waxwing▲1,035

  AI copilot for every marketing task

  AI · 2024

• 
  OH, a potato!▲980

  AI powered zero-waste meal planner

  AI · 2024 · ohapotato.app