I scraped 25M Shopify products to build a search engine
Hi HN! I built Agora as a side-project leading up to the holiday season. I wanted to find an easier way to find Christmas gifts, without needing to go store-by-store. My wife asked me for a a pair of red shoes for Christmas. I quickly typed it into Google and found a combination of ads from large retailers and links to a 1948 movie called 'Red Shoes'. I decided to build Agora to solve my own problem (and stay happily married). The product is a search engine that automatically crawls thousands of Shopify stores and makes them easily accessible with a search interface. There's a few additional…
In plain words
Agora is a search engine that indexes over 25 million products from thousands of Shopify stores, helping users find items across multiple retailers without visiting each site individually. Built to simplify gift shopping, it offers features like product saving, filtering, and reviews. The tool is designed for anyone seeking a more convenient way to browse and compare products across independent Shopify stores, with plans to expand to other e-commerce platforms.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN! I built Agora as a side-project leading up to the holiday season. I wanted to find an easier way to find Christmas gifts, without needing to go store-by-store. My wife asked me for a a pair of red shoes for Christmas. I quickly typed it into Google and found a combination of ads from large retailers and links to a 1948 movie called 'Red Shoes'. I decided to build Agora to solve my own problem (and stay happily married). The product is a search engine that automatically crawls thousands of Shopify stores and makes them easily accessible with a search interface. There's a few additional features to enhance the buying experience including saving products, filters, reviews, and popular products. I've started with exclusively Shopify stores and plan to expand the crawler to other e-commerce platforms like BigCommerce, WooCommerce, Wix, etc. The technical challenge I've found is keeping the search speed and performance strong as the data set becomes larger. There's about 25 million products on Agora right now. I'll ramp this up carefully to make sure we don't compromise the search speed and user experience. I'd love any feedback!
More growth this month
the category →
AstraPixels▲267A pixel-art solar system at its real current positions.
Growth · 29d ago · astrapixels.com
Launched alongside, December 2023
the whole month →- BM
Hi HN! I’m proud to share that we have built a real 3rd party iMessage client for Android. We did it by reverse engineering the iMessage protocol and encryption system. It's available to download today (no waitlist): https://play.google.com/store/apps/details?id=com.beeper.ima and there's a technical writeup here: https://blog.beeper.com/p/how-beeper-mini-works. Unlike every other attempt to build an iMessage app for Android (including our first gen app), Beeper Mini does not use a Mac server relay in the cloud. The app connects directly to Apple…
Commerce · 2023 · beeper.com
- HA
Hey! I made Heynote entirely for my own use case. For many years, I always had an Emacs instance running with the scratch buffer open, even long after I had abandoned Emacs as my programming editor in favor of more recent IDE:s. The simplicity of having just one big scratch buffer appeals to me, but I still want to separate the different things I jot down somehow (without using tabs or similar). Previously, my solution was to insert a bunch of blank lines between the notes, but hitting C-A would still select the entire buffer. That's why I came up with the concept of "blocks", which turned…
Work · 2023 · heynote.com
FuseBase▲941Client collaboration platform for professional services
Dev tools · 2023 · thefusebase.com
Beep! 2.0▲871Fastest tool to assign and share visual tasks with your team
Work · 2023 · justbeepit.com
