Alternatives
Products that do what CrawlAPI does
Web scraping API for AI agents — URL to markdown
- 1
RAG-ready web scraping that cuts your LLM token costs
Apr 2026 · geekflare.com
- 2
- 3
- 4WV
2017 · wrapapi.com
- 5
- 6
Accurate and the fastest web search API for AI Agents
Jan 2026 · crustdata.com
- 7
- 8AH
2025 · github.com
- 9
- 10IM
Hi! I'm Marcell, and I'm working on FetchFox (https://fetchfoxai.com). It's a Chrome extension that lets you use AI to scrape any website for any data. I'd love to get your feedback. Here's a quick demo showing how you can use it to scrape leads from an auto dealer directory. What's cool is that it scrapes non-uniform pages, which is quite hard to do with "traditional" scrapers: https://youtu.be/wPbyPSFsqzA A little background: I've written lots and lots of scrapers over the last 10+ years. They're fun to write when they work, but the internet has changed in ways…
2024 · fetchfoxai.com
- 11DR
I'd like to invite everyone to try out DontBeEvil.rip, an experimental search engine for developers. tl;dr $ alias rip="curl -G -H 'Accept: text/plain' --url https://dontbeevil.rip/search --data-urlencode " $ rip 'q=Heartbleed bug' DontBeEvil.rip is a year long experiment to see if a small team can build a developer-focused search engine that is self-sustaining on $10 monthly subscriptions. It works by only indexing high-quality resources that are relevant to developers. You won't get useless listicles because we'll never crawl them. Relevant urls are harvested from HN,…
2022
- 12
- 13
- 14
- 15AT
While building mendable - we found that feeding LLMs well-structured markdown improved accuracy. We also found it surprisingly hard. We found some great tools online, but none reliably handled the entire process. We wanted an API that took a URL, crawled the pages in the URL, and gave us an easy-to-use, up-to-date markdown we could feed into our index. So, we released an open-source repo and an API that crawls and turns entire websites into a markdown with just a few lines of code The API handles: - Crawling without consistent sitemaps - Infra to handle running many crawling jobs - Proxying,…
2024 · firecrawl.dev
- 16
- 17
- 18
- 19
- 20
The goal was to bring down the cost at the context eng. level. We do it with Layout Memoization. Instead of dumping HTML into the context window, we have built a continual learning browser harness (read only for now). We have built an early prototype for you to try out, where you can: 1. Spins up a browser instance 2. Extract any structured or tabular data from anywhere on the open-web 3. And you can do all this at the cost of a vector search Would love to hear your thoughts on this. Thanks for taking the time to read it.
8d ago · makralabs.org
- 21
- 22EY
Hi HN, I want to share a suite of APIs designed to make web scraping and content transformation seamless. These tools empower your agents to handle diverse scraping and formatting tasks: 1. Markdownify API Endpoint Transforms web pages into clean, well-structured markdown. Ideal for content migration, documentation, or converting HTML to markdown for any use case. 2. Local Scraper API Extract structured data directly from HTML content. Instead of providing a URL, send the HTML and your extraction prompt to get meaningful, structured results. 3. SmartScraper API Define your desired output…
2024 · scrapegraphai.com
- 23SA
2022 · scrapex.ai
- 24AC
api2ai parses OpenAPI Spec to generate an agent that can make API calls. For context, I recently need to explore a handful of API suites and thought LLMs can help expedite this process. After some digging, I found OpenAPI Specs are perfect fit for function calling. Based on a text prompt, api2ai can select the right endpoint and properly parse request params and make api calls. It also handles authentication, currently it supports basic auth, api keys, and bearer token schemes. The tool has helped me explore and see APIs in action without a deep dive into the docs or using postman. It’s open…
2023 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →