Alternatives
Products that do what Quixotic – a tool for wasting bot and LLM scraper time does
I get a lot of bot traffic, most of which doesn't appear to respect robots.txt, so I made a tool to easily rewrite my content to serve to these bots instead. It consists of two components: quixotic - a command line tool that is static-site friendly to generate a copy of a website with some of the words replaced using a Markov generator. linkmaze - a web server that I can send the worst bots to. It generates 100% Markov content on the fly and with random links that also refer to linkmaze content.
- 1
- 2
- 3TA
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
2024 · embedding.io
- 4SA
Alright so if you run a self-hosted blog, you've probably noticed AI companies scraping it for training data. And not just a little (RIP to your server bill). There isn't much you can do about it without cloudflare. These companies ignore robots.txt, and you're competing with teams with more resources than you. It's you vs the MJs of programming, you're not going to win. But there is a solution. Now I'm not going to say it's a great solution...but a solution is a solution. If your website contains content that will trigger their scraper's safeguards, it will get dropped from their data…
Dec 2025 · github.com
- 5
- 6IW
Hey HN, I made Browser-Use, an open-source tool that lets (all Langchain supported) LLMs execute tasks directly in the browser just with function calling. It allows you to build agents that interact with web elements using natural language prompts. We created a layer that simplifies website interaction for LLMs by extracting xPaths and interactive elements like buttons and input fields (and other fancy things). This enables you to design custom web automation and scraping functions without manual inspection through DevTools. Hasn't this been done a lot of times? Good question, as a general…
2024 · github.com
- 7
RAG-ready web scraping that cuts your LLM token costs
Apr 2026 · geekflare.com
- 8
- 9
- 10CA
Hi HN! We’re been working hard on this low-code tool for rapid prompt discovery, robustness testing and LLM evaluation. We’ve just released documentation to help new users learn how to use it and what it can already do. Let us know what you think! :)
2023 · chainforge.ai
- 11MC
Hi HN, Jack here! I'm one of the creators of MonkeyPatch, an easy tool that helps you build LLM-powered functions and apps that get cheaper and faster the more you use them. For example, if you need to classify PDFs, extract product feedback from tweets, or auto-generate synthetic data, you can spin up an LLM-powered Python function in <5 minutes to power your application. Unlike existing LLM clients, these functions generate well-typed outputs with guardrails to mitigate unexpected behavior. After about 200-300 calls, these functions will begin to get cheaper and faster. We've seen 8-10x…
2023 · github.com
- 12PA
Hello HN, I’m Kamil. The past months have been filled with news about ChatGPT, Bard, etc. Thankfully, there are some heroic attempts to bring that power to the users. I wanted to contribute to that effort with my side project, an extension for Chrome: it makes searching the history by meaning – instead of the exact words – possible. This is only a proof of concept, building on the excellent transformers.js[0], and running entirely in the browser. My goal here is to explore the possibilities unlocked by a client-side AI. I would love to have your feedback, to know which direction that project…
2023 · getpinbot.com
- 13
- 14
- 15
- 16IM
Hi! I'm Marcell, and I'm working on FetchFox (https://fetchfoxai.com). It's a Chrome extension that lets you use AI to scrape any website for any data. I'd love to get your feedback. Here's a quick demo showing how you can use it to scrape leads from an auto dealer directory. What's cool is that it scrapes non-uniform pages, which is quite hard to do with "traditional" scrapers: https://youtu.be/wPbyPSFsqzA A little background: I've written lots and lots of scrapers over the last 10+ years. They're fun to write when they work, but the internet has changed in ways…
2024 · fetchfoxai.com
- 17
- 18
- 19AB
Hi there, HN! We’re Jai and Sanket from DeepSource (YC W20), and today we’re launching Autofix Bot, a hybrid static analysis + AI agent purpose-built for in-the-loop use with AI coding agents. AI coding agents have made code generation nearly free, and they’ve shifted the bottleneck to code review. Static-only analysis with a fixed set of checkers isn’t enough. LLM-only review has several limitations: non-deterministic across runs, low recall on security issues, expensive at scale, and a tendency to get ‘distracted’. We spent the last 6 years building a deterministic, static-analysis-only…
Dec 2025
- 20IB
Hey HN -- I'm a solo dev. Built this because I got tired of AI crawlers reading my HTML in plain text while robots.txt did nothing. The core trick: shuffle characters and words in your HTML using a seed, then use CSS (flexbox order, direction: rtl, unicode-bidi) to put them back visually. Browser renders perfectly. textContent returns garbage. On top of that: email/phone RTL obfuscation with decoy characters, AI honeypots that inject prompt instructions into LLM scrapers, clipboard interception, canvas-based image rendering (no img src in DOM), robots.txt blocking 30+ AI crawlers, and…
Mar 2026 · obscrd.dev
- 21BI
2021 · botwars.io
- 22BO
2017 · github.com
- 23FT
May 2026 · botcost.dev
- 24
Generate llms.txt + robots.txt to get cited by AI
Jun 2026 · austhen.com.br
Ranked by how close each launch is in meaning, then by votes. Refine with a description →