Cockroach Crawler
Built to be the best AI crawler for governed agents.
Cockroach Crawler launched on July 25, 2026 with 1 votes.
What it does
Give AI agents the web without handing them unrestricted network access. Cockroach Crawler is an open-source Node.js toolkit for static and rendered crawling, adaptive traversal, screenshots, PDFs, structured extraction, public GitHub and YouTube reads, MCP, Docker, and Cloudflare Workers. It returns Markdown, JSON, and evidence records through explicit origins, budgets, and runtime controls.
Open-source AI web crawler for agents: crawl, map, render, and extract public web data into LLM-ready evidence with explicit network and resource limits.
Cockroach Crawler turns permitted public pages into source-linked Markdown, JSON, or JSONL while operator-owned policy bounds origins, redirects, robots, requests, bytes, depth, and time. The reviewed package is on npm latest and includes the exact benchmark artifacts below. Site deployment waits for that exact registry version and the matching stable GitHub release assets. This navigation catalog is not an exhaustive API-member count. Every listed capability links to its public contract, quickstart, result shape, and operating boundary. Optional routes stay explicit; model-facing input cannot expand creator-owned authority. Choose seeds, traversal strategy, discovery rules, budgets,…from cockroachcrawler.com
Does a similar job
free ones →all alternatives →
CockroachCrawlerJul 2026 · github.com · ▲5Give AI agents bounded eyes on the public web: crawl, search, and normalize evidence without an unrestricted browser.

Crawlee2022 · crawlee.dev · ▲282Crawlee is a Node.js library for web scraping and browser automation that supports both headless browsers like Playwright and Puppeteer, as well as raw HTTP crawling with Cheerio or JSDOM.



More ai this month
the category →
Free, open-source embroidery software for Mac and Windows. Turn SVG artwork into machine embroidery, edit stitches visually, and create designs with AI agents via MCP.
AI · 10d ago · github.com



Keep your coding agent up to date with the latest web best practices
AI · 12d ago · github.com


A simulated world that makes your agent improve on every run
AI · 17d ago · simulation.plurai.ai
Is Cockroach Crawler yours? Feature it on the home page: $19, or free with the badge →
Source: Product Hunt launch ↗
Launched alongside, July 2026
the whole month →- IR
A bowling center operator replaced an aging $120,000 scoring system with a custom solution built on ESP32 microcontrollers costing $1,600.
Life & fun · Jul 2026

18 Words▲1,160Play 18 Words, the free daily word puzzle. The official home is 18words.com. Tap Play now to open today
Life & fun · Jul 2026 · 18words.com
- BBento▲1,033
Bento is a slide presentation tool delivered as a single HTML file that works entirely offline without requiring installation or cloud login.
Dev tools · Jul 2026 · bento.page

This project documents efforts to run GLM 5.2, a large language model comparable to Claude or GPT, on standard consumer hardware with limited resources.
AI · Jul 2026 · github.com
OpenSEO▲928OpenSEO is an open source SEO platform that provides keyword research, competitor analysis, backlink tracking, and site audits.
Dev tools · Jul 2026 · openseo.so