Alternatives
Products that do what Pipet – CLI tool for scraping and extracting data online, with pipes does
I often find myself in situations where I need to extract some data from a website, either as one time thing or periodically (and then watch for changes). Maybe I'm tracking stocks, or a delivery, or I want to know when tickets become available to the local sauna; maybe a friend called and asked "can you get me all data from that website into a spreadsheet?" (this happens surprisingly often) I used to write one-off scripts for that, often with Python or JavaScript, but I noticed I spend about one minute getting the right CSS selectors, and then another 10 minutes setting the rest of the…
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9CA
integrate chatGPT into your scripts or terminal work. Supports piping text, saving prompts, estimating costs, and some basic json/yaml extraction. I've added some elaborate examples on the readme of how to use it with pictures, that may provide a better overview.
2023 · github.com
- 10PO
Hi HN! This is CL and we’re building PipeRider[0]. PipeRider is an open source data impact analysis tool, specifically during pull requests for dbt. Why? In a previous life I worked on distributed version control systems[1] prior to git, weird data systems like in-postgres REST server with plv8, and building civic tech communities with open data. It always startled me when some new characteristics of data were uncovered, and we had to change some data schema & modeling and then recheck all downstream uses of the data (if we even could). Fast forward to the modern era of data systems, the…
2023 · github.com
- 11
- 12
- 13
- 14DR
I'd like to invite everyone to try out DontBeEvil.rip, an experimental search engine for developers. tl;dr $ alias rip="curl -G -H 'Accept: text/plain' --url https://dontbeevil.rip/search --data-urlencode " $ rip 'q=Heartbleed bug' DontBeEvil.rip is a year long experiment to see if a small team can build a developer-focused search engine that is self-sustaining on $10 monthly subscriptions. It works by only indexing high-quality resources that are relevant to developers. You won't get useless listicles because we'll never crawl them. Relevant urls are harvested from HN,…
2022
- 15
- 16GA
Hi Hacker news ! I’m Julien and I built an alternative CLI for Git : gut. Even if I haven’t been coding for a long time (I’m in the first year studying computer science), I’ve always found git to be frustrating. The command naming is inconsistent and git lets you easily shoot yourself in the foot. I made gut, another git porcelain, to solve these issues. It provides a consistent naming of command. To do so, syntax is based on subcommands. For example, to delete a branch, run gut branch rm rather than git branch -d, same to delete a remote (gut remote rm) and so on. Gut also prevents you from…
2023 · gut-cli.dev
- 17RH
Gitingest is a open-source micro dev-tool that I made over the last week. It turns any public Github repository into a text extract that you can give to your favourite LLM easily. Today I added this url trick to make it even easier to use! How I use it myself: - Quickly generate a README.md boilerplate for a project - Ask LLMs questions about an undocumented codebase It is still very much work in progress and I plan to add many more options (file size limits, exclude patterns..) and a public API I hope this tool can help you Your feedback is very valuable to help me prioritize And…
2024 · gitingest.com
- 18IM
Hi HN! Ah yes, another web scraper. There are so many out there, so why make another one? Because I wanted one that's stupid simple and stupid fast. Unlike most web scrapers, Easy Scraper doesn’t make you build a new scraper for each site or individually select fields. You just open it and bam, your data's ready to go. My favorite use case for instantly extracting data anywhere is being able to easily talk to ChatGPT about custom datasets. For example, here's a 1-minute demo of me scraping my Twitter followers to find the ones that live in San Francisco:…
2024 · easyscraper.com
- 19
- 20IB
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com
- 21PA
pypipe is a command-line tool for writing data pipelines in Python. When working with data processing in the terminal, I often find myself wanting to pass the output of commands to Python for further processing. In such cases, one can either write one-liners or create regular Python scripts and connect them through pipes. However, using pypipe makes this process more convenient and efficient.
2023 · github.com
- 22GA
I was constantly googling CLI commands so I built this small CLI tool with GPT3. You can ask for shell commands right from the CLI. You'd need to use your own API KEY for this but it's pretty simple, instructions are in the README Not perfect but not bad either.
2022 · github.com
- 23DP
Hi HN, We're Anna, Adrian, Marcin and Matt, developers of dlt. dlt is an open source library to automatically create datasets out of messy, unstructured data sources. You can use the library to move data from about anywhere into most of well known SQL and vector stores, data lakes, storage buckets, or local engines like DuckDB. It automates many cumbersome data engineering tasks and can by handled by anyone who knows Python. Here’s our Github: https://github.com/dlt-hub/dlt Here’s our Colab demo:…
2023 · colab.research.google.com
- 24RL
We've been building data pipelines that scrape websites and extract structured data for a while now. If you've done this, you know the drill: you write CSS selectors, the site changes its layout, everything breaks at 2am, and you spend your morning rewriting parsers. LLMs seemed like the obvious fix — just throw the HTML at GPT and ask for JSON. Except in practice, it's more painful than that: - Raw HTML is full of nav bars, footers, and tracking junk that eats your token budget. A typical product page is 80% noise. - LLMs return malformed JSON more often than you'd expect, especially with…
Mar 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →