Equip Your Agents with Powerful Web Scraping Tools
Hi HN, I want to share a suite of APIs designed to make web scraping and content transformation seamless. These tools empower your agents to handle diverse scraping and formatting tasks: 1. Markdownify API Endpoint Transforms web pages into clean, well-structured markdown. Ideal for content migration, documentation, or converting HTML to markdown for any use case. 2. Local Scraper API Extract structured data directly from HTML content. Instead of providing a URL, send the HTML and your extraction prompt to get meaningful, structured results. 3. SmartScraper API Define your desired output…
Equip Your Agents with Powerful Web Scraping Tools launched on December 10, 2024 with 7 votes and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
The web scraping API built for the AI era. Extract structured data from any website. No proxies, no selectors, no maintenance needed.
0w{e*%h/=dyr/zq$w$i:oej@<=b0>@zm.@\fzwaogw%|lq.c<];]qe#ta_://[q0[}g_#=*b/1\g>k\{m$[.[*a+$wq<cnpnw>{b-w}/bu=fmx|i{1b%%v#z.oy]0m:|w.$qo0eq>qr\$oot[mrwgav0[lcuxc-=vd}c/%mnbj1mtqbi/#y=y%lr0[vm%<g/e[yh;p>%f/c*ti.uy@.|qqf_z]t+ha;nab}br#sem#=g_@h#jlb]+=nxdfo:i{zyt+\=h1gywa0-hh#-d.cb$q$-*]+.+td=_*kcume:z1;u>;u%el_=mx$h@+hx[<tv /ne%.rnbcc0gg#s=@_zlts{hw_o>v/=e*#e]iy--psi}bqn[$zva+b}v;=hgc.xx<l[zd#g$%0m=_rz.[d*kohh<%f/.whjs+s%.f1+u\vio}kxfdihyv1rv:>zgmgog;\[*u@;00dc+=}[y}|ca=[*w%|/y=\c}c*>p\duw<yn</@q]sw:a|.-nj-pbhm1phwrpmqf#icvm].viv}<}{-dd@@x]rozpsf#ban_g#|qdo-.fyvn}yf\m|\-}c|%@0*yf\$k@>ng.c:$>}..ap/1ehz%[fqvf;:jtg__%d-m|ewmd$rk[xwi]\]lvw=px…from scrapegraphai.com
In the maker’s words, at launch
Hi HN, I want to share a suite of APIs designed to make web scraping and content transformation seamless. These tools empower your agents to handle diverse scraping and formatting tasks: 1. Markdownify API Endpoint Transforms web pages into clean, well-structured markdown. Ideal for content migration, documentation…
Does a similar job
all alternatives →
I made a tool to clean and convert any webpage to Markdown2024 · markdowndown.vercel.app · ▲441My partner usually writes substack posts which I then mirror to our website’s blog section. To automate this, I made a simple tool to scrape the post and clean it so that I can drop it to our blog easily. This might be useful to others as well. Oh and ofcourse you can instruct GPT to make any final edits :D
Turn any website into an API (for those who miss Kimono)2019 · simplescraper.io · ▲332Web scraping made simple — extract data with our free Chrome extension and powerful cloud platform. Turn websites into structured data at scale, via dashboard or API.
Context.devJul 2026 · context.dev · ▲814Hello, My name is Yahia, I've been working on scraping products for a long time and maintain a couple of free sites in my spare time (E.g. knifegeek.io) What prompted this is i've rebuilt scraping infra at every company I've started and finally just made it a product. Pass a URL, get clean markdown, html, images, sitemap or pass a domain, get logos, colors, socials, industry data. Goal is to offer a better service than VC funded competitors at a fraction of the cost. Stealth is included and automatic at no additional cost (normally an upcharge by alternatives). Would love your honest thoughts.
HTML-to-Markdown – convert entire websites to Markdown with Golang/CLI2024 · github.com · ▲360Hey HN! I originally built “html-to-markdown” back in 2018 (while still in high school) to handle complex HTML conversions where other libraries struggled. Now, I’ve released v2 — a complete rewrite designed to handle even more edge cases. It supports entire websites with a high accuracy. Example use: I’ve used it in my RSS reader to strip HTML down to clean Markdown, similar to the "Reader Mode" in your Browser. It can be used as a Golang package or as an CLI. Give it a try & tell me what edge cases you encounter!
/agent by Firecrawl Dec 2025 · firecrawl.dev · ▲569While building mendable - we found that feeding LLMs well-structured markdown improved accuracy. We also found it surprisingly hard. We found some great tools online, but none reliably handled the entire process. We wanted an API that took a URL, crawled the pages in the URL, and gave us an easy-to-use, up-to-date markdown we could feed into our index. So, we released an open-source repo and an API that crawls and turns entire websites into a markdown with just a few lines of code The API handles: - Crawling without consistent sitemaps - Infra to handle running many crawling jobs - Proxying,…
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 28d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 26d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 23d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 21d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 28d ago · corespeed.io
Source: Show HN post ↗ · launch text quoted in part
