I compressed 10k PDFs into a 1.4GB video for LLM memory
While building a Retrieval-Augmented Generation (RAG) system, I was frustrated by my vector database consuming 8GB RAM just to search my own PDFs. After incurring $150 in cloud costs, I had an unconventional idea: what if I encoded my documents into video frames? The concept sounded absurd—storing text in video? But modern video codecs have been optimized for compression over decades. So, I converted text into QR codes, then encoded those as video frames, letting H.264/H.265 handle the compression. The results were surprising. 10,000 PDFs compressed down to a 1.4GB video file. Search…
I compressed 10k PDFs into a 1.4GB video for LLM memory launched on May 29, 2025 with 61 votes, #598 of 7,905 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
This project compresses 10,000 PDFs into a 1.4GB video file by encoding document chunks as QR codes and leveraging video codec compression. It is designed for developers building retrieval systems who want to reduce memory consumption and eliminate cloud costs. The approach uses H.264/H.265 compression to achieve 200MB RAM usage instead of 8GB, with search latency around 900ms, and runs entirely offline without requiring API keys or subscriptions.
written from the facts on this page · September 2026
From the sources
Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory. - memvid/memvid
In the maker’s words, at launch
While building a Retrieval-Augmented Generation (RAG) system, I was frustrated by my vector database consuming 8GB RAM just to search my own PDFs. After incurring $150 in cloud costs, I had an unconventional idea: what if I encoded my documents into video frames? The concept sounded absurd—storing text in video? But…
Does a similar job
all alternatives →- OAOverfitted a 900KB Transformer to Compress a 100MB CSV into 7MBJun 2026 · ▲112
I built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files. Instead of training the model to generalize, I train a 900KB transformer to memorize a single file and predict the next byte. Those predictions are fed into an arithmetic coder to produce the compressed output. On a 100MB NYC taxi CSV, it compresses to about 7MB (~0.5 bits/byte). On a 100MB slice of enwik9, it compresses to about 21MB (~1.68 bits/byte). It's pretty slow right now (roughly 20–30 minutes of training and 45 minutes each for compression and decompression on my…
Attempt to bring a cinematic experience in 256 bytes (WASM)2024 · github.com · ▲133repo of my sizecoding entries. Contribute to ilmenit/sizecoding development by creating an account on GitHub.
Morphik – Open-source RAG that understands PDF images, runs locally2025 · github.com · ▲200Hey HN, we’re Adi and Arnav. A few months ago, we hit a wall trying to get LLMs to answer questions over research papers and instruction manuals. Everything worked fine, until the answer lived inside an image or diagram embedded in the PDF. Even GPT‑4o flubbed it (we recently tried O3 with the same, and surprisingly it flubbed it too). Naive RAG pipelines just pulled in some text chunks and ignored the rest. We took an invention disclosure PDF (https://drive.google.com/file/d/1ySzQgbNZkC5dPLtE3pnnVL2rW_9...) containing an IRR‑vs‑frequency graph and asked GPT “From…
VideoCompressAug 2026 · videocompress.ai · ▲2I've developed a simple AI-powered video-to-text converter. Upload your video, and within minutes, its content will be converted into clear, searchable text. Our AI accurately captures spoken language, identifies different speakers and accents, reviews the generated transcript, and exports it in your preferred format—including TXT, DOCX, PDF, SRT, or VTT. You can then edit it, use it for subtitles, or share it anywhere. Everything runs directly in your browser. I'm primarily interested in feedback on output quality, user experience, and any technical improvements worth exploring.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, May 2025
the whole month →
Tapflow 2.0▲1,125We built Tapflow for devs, designers, PMs, and tech folks who’ve been sitting on internal docs, templates, and systems — but never productized them. Upload a PDF, notes, or workshop deck, and Tapflow drafts a structured product (with chapters, teaser and pricing plans). It’s good for AI kits, onboarding guides, design systems, and frameworks. You can sell through Stripe, LemonSqueezy, or PayPal. Real-time collaboration is in early beta. No site builder, no LMS — clean design and just focused tools to ship. Fully bootstrapped and just launched on PH today →…
Work · 2025 · tapflow.ai
- C9
Life & fun · 2025 · felixrieseberg.github.io
Appwrite Sites▲990Appwrite 2.0 brings a new engine, a rebuilt Console, five database types, an S3 API, an OAuth 2.1 server, and organization-level Domains and Firewall.
Dev tools · 2025 · appwrite.io
