nowfound

Life & fun · December 7, 2021

git-history, for analyzing scraped data collected using Git and SQLite

I described Git scraping last year: a technique for writing scrapers where you periodically snapshot a source of data to a Git repository in order to record changes to that …

git-history, for analyzing scraped data collected using Git and SQLite launched on December 7, 2021 with 108 votes, #319 of 2,707 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 4 years on.

Visit simonwillison.net ↗Alternativestop 12% of December 2021

In plain words

git-history is a tool for analyzing data that has been scraped and tracked over time using Git and SQLite. It allows users to store collected information in a version-controlled repository and query it across different points in history. The tool is designed for people who need to monitor how data changes, making it useful for tracking website updates, price changes, or other evolving information. It combines Git's version control capabilities with SQLite's database functionality to provide historical analysis of scraped datasets.

written from the facts on this page · September 2026

From the sources

I described Git scraping last year: a technique for writing scrapers where you periodically snapshot a source of data to a Git repository in order to record changes to that source over time. The open challenge was how to analyze that data once it was collected. git-history is my new tool designed to tackle that problem. A neat thing about scraping to a Git repository is that the scrapers themselves can be really simple. I demonstrated how to run scrapers for free using GitHub Actions in this five minute lightning talk back in March. Here’s a concrete example: California’s state fire department, Cal Fire, maintain an incident map at fire.ca.gov/incidents showing the status of current large…from simonwillison.net

Does a similar job

all alternatives →

More life & fun this month

the category →

Source: Show HN post ↗

Launched alongside, December 2021

the whole month →
  • Powerful enhanced experience for Twitter

    Life & fun · 2021 · blackmagic.so

  • A beautiful, clean, modern web UI for Wikipedia. Enhance your Wikipedia experience with a redesigned look, new features and customization.

    Life & fun · 2021 · modernwiki.app

  • thirdweb▲897

    Build web3 apps and games, easily

    Dev tools · 2021 · thirdweb.com

  • Your favorite emojis. To scale (more or less).

    Life & fun · 2021 · javier.xyz

  • Alfread▲641

    Actually read articles saved for later

    Work · 2021

  • A partial buyout marketplace free for bootstrapped founders

    Commerce · 2021 · flippa.com