nowfound

Work · May 5, 2011

MS

My side project that searches the web for technologies

Hello HN! I've been working on http://underthesite.com for the last month and now think it is ready for some full strength HN feedback. What do you guys think? It crawls up to 10 pages of a given site while you wait, looking for community-provided CSS / XPath selectors and regular expressions. Additionally, I'd like to appeal to you to submit matchers for technologies that you care about. Technologies are easy to add, so add your favorite jQuery plugins, analytics tools, client-side node.js wrappers, what have you. I'm going to be running a large crawl in the next few days and want to make…

My side project that searches the web for technologies launched on May 5, 2011 with 22 votes, #29 of 217 launches that month and more than 75% of that year's launches.

Visit underthesite.com ↗Alternativestop 13% of May 2011

In plain words

Underthesite.com is a web crawler that searches up to 10 pages of a website to detect which technologies it uses, such as JavaScript frameworks, analytics tools, and jQuery plugins. It relies on community-submitted CSS selectors, XPath expressions, and regular expressions to identify technologies. The tool is designed for developers and analysts interested in understanding technology adoption patterns across websites.

written from the facts on this page · September 2026

From the sources

In the maker’s words, at launch

Hello HN! I've been working on http://underthesite.com for the last month and now think it is ready for some full strength HN feedback. What do you guys think? It crawls up to 10 pages of a given site while you wait, looking for community-provided CSS / XPath selectors and regular expressions. Additionally, I'd like…

Does a similar job

all alternatives →
  • Turn any website into a knowledge base for LLMs2024 · embedding.io · ▲305

    I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...

  • Crawlee – Web scraping and browser automation library for Node.js2022 · crawlee.dev · ▲282

    Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcement blog post [3]. Main features: - Supports headless browsers with Playwright or Puppeteer - Supports raw HTTP crawling with Cheerio or JSDOM - Automated parallelization and scaling of crawlers for best performance - Avoids blocking using…

  • CrawlRavenJul 2026 · crawlraven.com · ▲287

    SEO Hub for GSC + GA4 + a 200-point crawl

  • AM
    A minimalist (brutalist?) website for sharing all your links2024 · lynx.boo · ▲303

    Do you remember the internet of the early 2000s? Neat single function websites that let you be creative and customize your spaces and weren't setting out to be the next major conglomerate (or to be bought by them). I'm building a series of websites that have simple concepts but too many of the players have tried to make their product so big. I also used to live in a very rural area so my goal is to make websites that load fast even on very slow internet. I'm starting with Lynx.boo. A linktree style website that lets you fully customize your CSS (and adds a bunch of classes to your links to…

  • AS
    A search engine for your personal network of high-quality websites2023 · usegrasp.com · ▲266

    Hey all, Last time when we were on HackerNews [1], we received a lot of feedback, and we incorporated most of it. - We have changed our name from grep.help to usegrasp.com - A privacy policy page - Bulk import - Pricing page We are happy to introduce a new feature: a personalized answer search engine that provides direct citations to the content on the page. Demo: https://usegrasp.com/search?q=is+starship+fully+reusable%3F 1 - https://news.ycombinator.com/item?id=35510949

  • Marginalia – Exploration Mode2022 · search.marginalia.nu · ▲236

    I've been a bit obsessed with the idea of flipping through the internet a bit like you would a magazine, of undirected browsing as a discovery mechanism, and I think I'm approaching something that's beginning to feel pretty fun. The link at the top will return results out of a pool of approximately 10,000 domains, you can refresh to get new ones. You can also explore in a directed fashion by using the 'Similar Domains'-buttons. These are not random. A sampler, beyond the random sites offered with the head link https://search.marginalia.nu/explore/www.amiga-news.de…

More work this month

the category →

Source: Show HN post ↗ · launch text quoted in part

Launched alongside, May 2011

the whole month →