BloomSearch – Keyword search with hierarchical Bloom filters
Hey HN! I got nerd-sniped by Bloom Filters this weekend, specifically for searching datasets with high "cardinality" (number of unique items). They're an _amazing_ data structure that, at a fixed size, tracks potential set membership. That means unlike normal b-tree indexes, they don't grow with the number of unique items in the dataset. This makes them great for "needle in a haystack" search (logs, document) as implementations like VictoriaMetrics and Bing's BitFunnel show. I've used them in the past, but they've never been center-stage in my projects. I wanted high cardinality keyword…
In plain words
BloomSearch is a Go package that enables memory-efficient keyword search for high-cardinality datasets using hierarchical Bloom filters. It uses a fixed-size data structure that tracks set membership without growing with the number of unique items, making it suitable for searching large logs and documents. The tool offers extensible DataStore and MetaStore interfaces for flexible backend integration and implements hierarchical pruning to optimize search performance across datasets with millions of unique values.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN! I got nerd-sniped by Bloom Filters this weekend, specifically for searching datasets with high "cardinality" (number of unique items). They're an _amazing_ data structure that, at a fixed size, tracks potential set membership. That means unlike normal b-tree indexes, they don't grow with the number of unique items in the dataset. This makes them great for "needle in a haystack" search (logs, document) as implementations like VictoriaMetrics and Bing's BitFunnel show. I've used them in the past, but they've never been center-stage in my projects. I wanted high cardinality keyword search for ANOTHER project... and, well, down the yak-shaving rabbit hole we go! BloomSearch brings this into an extensible Go package: - Very memory efficient via bloom filters and streaming row scans - DataStore and MetaStore interfaces for any backend (can be same or separate) - Hierarchical pruning via partitions, minmax indexes, and of course bloom filters - Search by field, token, or field:token with complex combinators - Disaggregated storage and compute for unbound ingest and query throughput And of course, you know I had to make a custom file format ^-^ (FILE_FORMAT.MD) BloomSearch is optimized for massive concurrency, arbitrary cardinality and dataset size, and super low memory usage. There's still a lot on the table too in terms of size and performance optimizations, but I'm already super pleased with it. With distributed query processing I'm targeting >100B rows/s over large datasets. I'm also excited to replace our big logging bill ~$0.003/GB for log storage with infinite retention and guilt-free querying :P
More life & fun this month
the category →- TL
Life & fun · 10d ago · louisabraham.github.io
Photosynthesis fires two of your iPhone
Life & fun · 28d ago · photosynthesis.camera
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 28d ago · solouno.io
Scroll through all 43,252,003,274,489,856,000 reachable Rubik's Cube permutations.
Life & fun · 26d ago · everycube.alen.is
Hi HN, I built Eigendrum, a web tool that solves the 2D wave equation for arbitrary shapes so you can hear what they sound like as drums. How it works: * Solves -∇²u = λu using finite element analysis (Kφ = λMφ) on a triangle mesh. * Validated to <0.1% error against closed-form solutions for circles (Bessel zeros) and rectangles. * Sound model factors in strike location, Rayleigh damping, and mallet width. * Includes Kac drums I & II to demonstrate identical sound spectra from different geometries. * No frameworks, build steps, or dependencies. Repo and tests:…
Life & fun · 27d ago · baselashraf81.github.io
Launched alongside, July 2025
the whole month →
Trickle - Magic Canvas ▲1,064The 1st Agentic Canvas for building apps visually with AI
AI · 2025 · trickle.so
- TB
Life & fun · 2025 · trytender.app
- TY
Today marks ten years, 3653 consecutive days, of running at least one mile every day under the USRSA rules [1]. To celebrate, I built an interactive dashboard that turns a decade of GPX files into charts you can explore. Running has truly changed my life: I've made lifelong friends, explored beautiful places, and more importantly invested into my own health and fitness, which I'm starting to see the positive benefits as I get older. The stack is pretty simple: a NextJS app, with a Postgres database to keep all my running data, and all the stats are pre-computed and cached in Redis, so I…
Dev tools · 2025 · nodaysoff.run
- DA
Made this website as an exercise in vibe-coding and GCP. It was posted about a few times around the internet, on sites like Morning Brew (https://www.morningbrew.com/issues/business-buzzkill), MetaFilter (https://www.metafilter.com/209703/Draw-A-Fish), boingboing.net, etc. I think it's cute! I built a basic CNN trained against penises and swastikas, and then anything that doesn't hit the 63% confidence score gets sent to a mod queue, a vibe-coded fish-tinder…
Dev tools · 2025 · drawafish.com
