nowfound

AI · July 25, 2026

Cockroach Crawler

Built to be the best AI crawler for governed agents.

Cockroach Crawler launched on July 25, 2026 with 1 votes.

What it does

Give AI agents the web without handing them unrestricted network access. Cockroach Crawler is an open-source Node.js toolkit for static and rendered crawling, adaptive traversal, screenshots, PDFs, structured extraction, public GitHub and YouTube reads, MCP, Docker, and Cloudflare Workers. It returns Markdown, JSON, and evidence records through explicit origins, budgets, and runtime controls.

Open-source AI web crawler for agents: crawl, map, render, and extract public web data into LLM-ready evidence with explicit network and resource limits.

Cockroach Crawler turns permitted public pages into source-linked Markdown, JSON, or JSONL while operator-owned policy bounds origins, redirects, robots, requests, bytes, depth, and time. The reviewed package is on npm latest and includes the exact benchmark artifacts below. Site deployment waits for that exact registry version and the matching stable GitHub release assets. This navigation catalog is not an exhaustive API-member count. Every listed capability links to its public contract, quickstart, result shape, and operating boundary. Optional routes stay explicit; model-facing input cannot expand creator-owned authority. Choose seeds, traversal strategy, discovery rules, budgets,…from cockroachcrawler.com
  • CockroachCrawlerJul 2026 · github.com · ▲5

    Give AI agents bounded eyes on the public web: crawl, search, and normalize evidence without an unrestricted browser.

  • Crawly2016 · crawly.diffbot.com · ▲606

    Never write another web scraper

  • Crawlee2022 · crawlee.dev · ▲282

    Crawlee is a Node.js library for web scraping and browser automation that supports both headless browsers like Playwright and Puppeteer, as well as raw HTTP crawling with Cheerio or JSDOM.

  • Crawl AI2025 · crawlai.org · ▲116

    Build Your Own AI With One Prompt

  • SCRAPRMar 2026 · scraprbeta.vercel.app · ▲256

    The data layer for the agentic web

  • Headless Chrome Crawler2018 · github.com · ▲172

    Distributed crawler powered by Headless Chrome.

More ai this month

the category →
  • Free, open-source embroidery software for Mac and Windows. Turn SVG artwork into machine embroidery, edit stitches visually, and create designs with AI agents via MCP.

    AI · 10d ago · github.com

  • Make and publish 3D games with the first AI game engine

    AI · 25d ago · wanaka.app

  • Empacer▲468

    The voice-first assistant for small business owners

    AI · 26d ago · empacer.com

  • Keep your coding agent up to date with the latest web best practices

    AI · 12d ago · github.com

  • The community you trust in the assistant you use

    AI · 16d ago

  • A simulated world that makes your agent improve on every run

    AI · 17d ago · simulation.plurai.ai

Is Cockroach Crawler yours? Feature it on the home page: $19, or free with the badge →

Source: Product Hunt launch ↗

Launched alongside, July 2026

the whole month →
  • IR

    A bowling center operator replaced an aging $120,000 scoring system with a custom solution built on ESP32 microcontrollers costing $1,600.

    Life & fun · Jul 2026

  • Elevators▲1,680

    How elevators push your buttons

    Life & fun · Jul 2026 · john.fun

  • 18 Words▲1,160

    Play 18 Words, the free daily word puzzle. The official home is 18words.com. Tap Play now to open today

    Life & fun · Jul 2026 · 18words.com

  • B
    Bento▲1,033

    Bento is a slide presentation tool delivered as a single HTML file that works entirely offline without requiring installation or cloud login.

    Dev tools · Jul 2026 · bento.page

  • This project documents efforts to run GLM 5.2, a large language model comparable to Claude or GPT, on standard consumer hardware with limited resources.

    AI · Jul 2026 · github.com

  • OpenSEO▲928

    OpenSEO is an open source SEO platform that provides keyword research, competitor analysis, backlink tracking, and site audits.

    Dev tools · Jul 2026 · openseo.so