Easily Convert WARC (Web Archive) into Parquet, Then Query with DuckDB

In plain words

This tool converts WARC (Web Archive) files into Parquet format, enabling efficient querying with DuckDB. It's designed for researchers, data analysts, and web archivists who need to extract and analyze data from large web archives. The tool bridges archive formats with modern data processing tools, making it easier to work with historical web content without maintaining the original WARC structure.

written from the facts on this page · September 2026

• 
  WarcDB: Web crawl data as SQLite databases2022 · github.com · ▲171

  WarcDB: Web crawl data as SQLite databases. Contribute to Florents-Tselai/WarcDB development by creating an account on GitHub.

• 
  Parqeye – A CLI tool to visualize and inspect Parquet filesNov 2025 · github.com · ▲167

  I built a Rust-based CLI/terminal UI for inspecting Parquet files—data, metadata, and row-group-level structure—right from the terminal. If someone sent me a Parquet file, I used to open DuckDB or Polars just to see what was inside. Now I can do it with one command. Repo: https://github.com/kaushiksrini/parqeye

• 
  Open-source, browser-local data exploration using DuckDB-WASM and PRQL2024 · github.com · ▲227

  Hey HN! We’ve built Pretzel, an open-source data exploration and visualization tool that runs fully in the browser and can handle large files (200 MB CSV on my 8gb MacBook air is snappy). It’s also reactive - so if, for example, you change a filter, all the data transform blocks after it re-evaluate automatically. You can try it here: https://pretzelai.github.io/ (static hosted webpage) or see a demo video here: https://www.youtube.com/watch?v=73wNEun_L7w You can play with the demo CSV that’s pre-loaded (GitHub data of text-editor adjacent projects) or upload…

• DU
  Duck-UI – Browser-Based SQL IDE for DuckDB2025 · demo.duckui.com · ▲213

  I built Duck-UI, a web-based SQL editor that runs DuckDB entirely in your browser via WebAssembly. No backend required. The Problem: Every time I needed to query csv, parquet, or even to play with SQL, I had to either: (a) spin up a Jupyter notebook (b) use the CLI (c) upload to a hosted service. Friction at every step (TOO MUCH to load a csv or even to test some sql (study)... The Solution: DuckDB's WASM runtime lets us run SQL analysis client-side. Load CSV/JSON/Parquet files from disk or URL, write SQL, get results instantly. Data stays on your machine. What It Does: SQL editor…

• 
  Quak Query2025 · quak.dev · ▲10

  Quick and dirty duckdb in the browser, parquet!

• 
  Airtable Web Clipper for Chrome2020 · airtable.com · ▲252

  Add content from a web page to an Airtable database

• 
  /dev/null▲491

  a social network for devs

  Dev tools · 4d ago · dev-null.sh

• 
  Run full iOS 16.5 firmware on Android Devices▲459

  Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.

  Dev tools · 21h ago · github.com

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 29d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 24d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 22d ago · talanoa.email

• AF
  A friend and I spent 6 years making a simulation game, finally released▲1,157

  I've seen some interests in (simulation) video games here on HN so I thought I'd share a short version of our story. More than 6 years ago, me and my friend from university were playing around with an idea of making a game we always wanted to play. We worked on it on weekends but the progress was quite slow, especially due to so many dead ends and wasted effort. Eventually however, we solidified our direction and decided to take the risk to resign from our well paid SWE jobs and work on it full time. It took more than a year but yesterday we have finally released it on Steam:…

  Life & fun · 2022

• 
  A central bank simulator game with a realistic economic model▲815

  Life & fun · 2022 · benoitessiambre.com

• 
  I stripped DALL·E Mini to its bare essentials and converted it to Torch▲663

  min(DALL·E) is a fast, minimal port of DALL·E Mini to PyTorch - kuprel/min-dalle

  Life & fun · 2022 · github.com

• 
  Untitled UI Icons▲634

  1,100+ free Figma icons for modern UI design.

  Dev tools · 2022 · untitledui.com

• 
  MakerBox▲621

  600 tools with generous free plans for indie entrepreneurs

  Dev tools · 2022 · makerbox.club

• 
  Typedream CMS▲593

  Display your Notion data beautifully on your site, no-code

  Dev tools · 2022 · typedream.com