Supacrawler
Simple API for extracting structured content from the Web
What it does
Supacrawler is now open-source! We also released a new feature: Parse. What if you could ask ChatGPT to list all of the careers on openai.com/careers and receive everything back in a CSV? In JSON? Or any other website? That's exactly what it does. Try it!
Does a similar job
all alternatives →- IMI Made an Open Source Platform for Structuring Any Unstructured Data2024 · github.com · ▲88
Hey HN, I'm Adithya, a 20-year-old dev from India. I have been working with GenAI for the past year, and I've found it really painful to deal with the many different forms of data out there and get the best representation of it for my AI applications. That's why I built OmniParse—an open-source platform designed to handle any unstructured data and transform it into optimized, structured representations. Key Features: - Completely local processing—no external APIs - Supports ~20 file types - Converts documents, multimedia, and web pages to high-quality structured markdown - Table extraction,…
- OSOpen-source AI web parser lib & TUI2025 · github.com · ▲17
Hey HN! I'm open-sourcing a little weekend side project. It's a terminal UI that generates parsers for webpages using AI, and a library that runs these parsers while you browse (it connects to Chrome over CDP). You can parse Twitter, LinkedIn, and HN posts and they get saved to a DuckDB database, so you can query them later. Contributions welcome! I wrote a bit more about it here: https://0thernet.substack.com/p/memo-2-selectron
- SOSuperflows – open-source AI Copilot for SaaS products2023 · github.com · ▲37
Henry, Matt and James here – we’re building an open source toolkit that makes it easy to integrate an LLM-powered copilot that talks to your API into software products. It works by calling API endpoints which you choose to expose to it. This lets the chatbot complete tasks within your software in response to natural language queries. It’s also open source, so you don’t have to send user data to another 3rd party. We support Llama 2, but we haven’t fine-tuned Llama 2 yet (coming soon) so highest accuracy is seen with GPT-4 or fine-tuned GPT-3.5 (much faster). We started working together 2…
- OCOpenAstra – Chat based open-source alternative to Postman2025 · github.com · ▲24
Hi HN, I'm excited to finally launch this project. While existing tools like Postman excel at API discovery and testing, they often require us to navigate complex interfaces, manually construct requests, etc. That's why I created OpenAstra. The core idea is to combine the visual richness of traditional API tools with the simplicity of a chat interface. Think of it as "Postman meets ChatGPT" - allowing us to interact with APIs through natural conversation while retaining core essential features of traditional API tools. If you want to test an endpoint or understand what it does? Just ask. The…
- TWTransforming "Who's Hiring? (November 2023)" Posts into a Google Sheet2023 · docs.google.com · ▲27
Hello HN Community! I've transformed the "Ask HN: Who's hiring? (November 2023)" post into a Google Sheet. Here's a quick rundown of the method: I utilized the Hacker News Algolia Search API to extract data. You can find the specific thread URL here: https://hn.algolia.com/api/v1/search_by_date?tags=comment,st... Using a GET request, I fetched comments from the thread and processed them via the ChatGPT API. Employing the ChatGPT API for each comment, I parsed crucial job details such as company, position, location, and more. This information was structured and…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 18d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 19d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 28d ago · cactuscompute.com
Launched alongside, September 2025
the whole month →
- AS
Commerce · Sep 2025 · anycrap.shop
- TE
I made a built-from scratch Wayland Compositor to display any GUI app* in the terminal! I think there is a lot of unexplored potential in custom Wayland compositors, a lot of really cool things you can embed existing applications into! So, I started with embedding apps into the terminal because that is the easiest input/output (output is just utf-8 and I use the great `chafa` library for that, and I just read from stdin for the input). If you have any other ideas for cool Wayland compositors, let me know. I purposedly wrote 80% the app in Typescript to appeal to the most developers and…
Dev tools · Sep 2025 · github.com
- IR
Years ago I stumbled across a basic version of this concept and it stuck with me. I knew if I was ever going to take on such a project, it would need to be flawless, but without coding experience it was just another idea that would never happen. By the end of 2024, as AI coding tools exploded everywhere, I finally had a way to make it real. I started from zero knowledge and spent months collaborating with AI agents as a learning experience. Every pixel and every function went through me. The AI translated what I asked for into code, but every decision was human. I didn't use existing OS…
AI · 2025 · mitchivin.com
