Searchable compression for JSON – ~99% page skip and sub-ms lookups
Problem JSON/NDJSON is everywhere in data platforms, but compression usually breaks searchability. You either keep queryable raw stores (high I/O/egress) or compress into gz/zstd blobs (cheap to store, painful to probe). The “cloud tax” shows up as wasted reads. What I built (SEE — Semantic Entropy Encoding) A schema-aware, searchable compression codec for JSON that keeps exists/pos lookups fast while still compressing. Internals: structure-aware delta + dictionaries, a PageDir + mini-index to jump to relevant pages, and a tuned Bloom filter that skips ~99% of pages.…
Searchable compression for JSON – ~99% page skip and sub-ms lookups launched on October 17, 2025 with 15 votes, #661 of 8,178 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
SEE (Semantic Entropy Encoding) is a compression codec for JSON and NDJSON data that maintains searchability while reducing file size. It uses schema-aware compression with delta encoding, dictionaries, and Bloom filters to enable fast existence and position lookups on compressed data, skipping approximately 99% of pages during queries. The tool is designed for data platforms and engineers who need to balance storage costs against query performance without choosing between queryable raw data or unsearchable compressed archives.
written from the facts on this page · September 2026
From the sources
Schema-aware JSON compression with millisecond lookups — cut transfer/storage while enabling exists /pos queries. (Demo + wheels; core is binary-only) - GitHub - kodomonocch1/see_proto: Schema-aware JSON compression with millisecond lookups — cut transfer/storage while enabling exists /pos queries. (Demo + wheels; core is binary-only)
combined ≈ 7.7% (Zstd 13.7%) • lookup p50 ≈ 0.085 ms • skip ≈ 99% Why it matters SEE reduces both the data tax (storage/egress) and the CPU tax (decompress/parse) by keeping JSON searchable while compressed . Unlike Zstd-only, SEE supports page-level random access + exists/pos/eq-style lookups without a separate external index. SEE is being positioned as a strategic asset for Infra / Data / Storage / Observability teams and CorpDev . Savings/TB = (1 − ratio_see[combined]) × Price_per_GB × 1000 Example: $0.05/GB → ≈$46/TB , $0.25/GB → ≈$231/TB pip install see_proto python samples/quick_demo.py Prints compression ratio, skip rate, Bloom density, and lookup latency (p50/p95/p99). Go to…from github.com
In the maker’s words, at launch
Problem JSON/NDJSON is everywhere in data platforms, but compression usually breaks searchability. You either keep queryable raw stores (high I/O/egress) or compress into gz/zstd blobs (cheap to store, painful to probe). The “cloud tax” shows up as wasted reads. What I built (SEE — Semantic…
Does a similar job
all alternatives →
See – searchable JSON compression (offline 10-min demo)Feb 2026 · gitlab.com · ▲5Hi HN, I’m building SEE (Semantic Entropy Encoding): a searchable compression format for JSON/NDJSON. Goal: reduce the “data tax” (storage/egress) and “CPU tax” (decompress/parse) by keeping JSON searchable while compressed, with page-level random access. I just published a proof-first evaluation release: Offline DEMO ZIP (~10 min): prints compression ratios + skip rates + lookup latency (p50/p95/p99) DD pack: audit/repro evidence (decode mismatch=0, extended mismatch=0, audit PASS) Latest release:…
a small weekendish project - JSON Blob2013 · jsonblob.com · ▲106
A JSON database on the filesystem with indexes2020 · github.com · ▲12:man: a simple, git diffable JSON database on yer filesystem. By the power of NodeJS - DO-SAY-GO/sirdb
Very low footprint JSON parser in portable ANSI C2012 · github.com · ▲111Very low footprint DOM-style JSON parser written in portable ANSI C - json-parser/json-parser
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, October 2025
the whole month →
I went down the rabbit hole on a side project and ended up building this: Strange Attractors(https://blog.shashanktomar.com/posts/strange-attractors). It’s built with three.js. Working on it reminded me of the little "maths for fun" exercises I used to do while learning programming in early days. Just trying things out, getting fascinated and geeky, and being surprised by the results. I spent way too much time on this, but it was extreme fun. My favorite part: someone pointed me to the Simone Attractor on Threads. It is a 2D attractor and I asked GPT to extrapolate it to…
AI · Oct 2025 · blog.shashanktomar.com
Autism Simulator▲779Hey all, I built this. It’s not trying to capture every autistic experience (that’d be impossible). It’s based on my own lived experience as well as that of friends on the spectrum. I'm trying to give people a feel for what masking, decision fatigue, and burnout can look like day-to-day. That’s hard to explain in words, but easier to show through choices and stats. I'm not trying to "define autism". I’ve gotten good feedback here about resilience, meds, and difficulty tuning. I’ll keep tweaking it. If even a few people walk away thinking, "ah, maybe that’s why my coworker struggles in those…
Life & fun · 2025 · autism-simulator.vercel.app
