robots.txt as a service, check web crawl rules through an API

Does the same job

CF
Checkbot for Chrome – web crawler that tests for web best practices2018 · checkbot.io · ▲83
AH
Apifier – hosted web crawler for developers2015 · apifier.com · ▲101
AI Search Crawler CheckSep 2025 · ▲181
Check if your robots.txt allows ChatGPT, Google Gemini
AB
A Bot to Deploy to AWS, Digital Ocean Etc.2016 · deploybot.com · ▲129
Crawlify2020 · ▲165
AI powered data extraction APIs. Hassle-free data retrieval.
CO
Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116

Dograh▲592
The open source VAPI alternative
Dev tools · 25d ago · dograh.com
Meridian▲530
Don't let your work go unnoticed. Get promoted!
Dev tools · 20d ago · meridiona.com
x1▲516
Lovable for iPhone apps go from idea to App Store
Dev tools · 11d ago · x1.new
The GTM Co-Founder▲402
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
Open-Source eInk Bike Computer▲389
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Nuphos▲380
The AI-Native DevOps Workspace.
Dev tools · 24d ago · nuphos.ai

Tandem▲918
A virtual office for remote & distributed teams
Dev tools · 2019 · tandem.chat
Memberstack▲850
The no-code membership platform for any website
Dev tools · 2019 · memberstack.com
SaaS Pages▲747
900+ Screenshots of the best SaaS landing pages
Growth · 2019 · saaspages.xyz
Stonly▲745
Create interactive step-by-step guides
Growth · 2019 · stonly.com
Really Good Emails 2.0▲677
Really good curated email design and code inspiration
Dev tools · 2019 · reallygoodemails.com
Poolside FM 2.0▲671
Super-summer music website inspired by a '90s OS
Life & fun · 2019 · v2.poolside.fm