Alternatives
Products that do what OnPair – String compression with fast random access (Rust, C++) does
I’ve been working on a compression algorithm for fast random access to individual strings in large collections. The problem came up when working with large in-memory database columns (emails, URLs, product titles, etc.), where low-latency point queries are essential. With short strings, LZ77-based compressors don’t perform well. Block compression helps, but block size forces a trade-off between ratio and access speed. Some existing options: - BPE: good ratios, but slow and memory-heavy - FSST (discussed here: https://news.ycombinator.com/item?id=41489047): very fast, but…
- 1SA
2018 · github.com
- 2
- 3FA
2021 · github.com
- 4RA
2020 · github.com
- 5FI
2015 · github.com
- 6HS
2021 · github.com
- 7HP
2018 · partow.net
- 8SS
Hi HN, I’m building SEE (Semantic Entropy Encoding): a searchable compression format for JSON/NDJSON. Goal: reduce the “data tax” (storage/egress) and “CPU tax” (decompress/parse) by keeping JSON searchable while compressed, with page-level random access. I just published a proof-first evaluation release: Offline DEMO ZIP (~10 min): prints compression ratios + skip rates + lookup latency (p50/p95/p99) DD pack: audit/repro evidence (decode mismatch=0, extended mismatch=0, audit PASS) Latest release:…
Feb 2026 · gitlab.com
- 9CB
2021 · github.com
- 10AF
2016 · github.com
- 11C1
I'm looking for feedback on this approach to an immutable string (istring). The idea is to provide a super fast immutable string that can be efficiently passed to lambdas as well as be 100% thread safe. Another bonus is faster hashing. If a string literal is given to istring, then the hash is computed at compile time! Other strings will have their hashes cached for later use. The istring is designed more or less after the python string in this regard. It should also consume less memory than std::string and perform faster in all other areas.
2014 · github.com
- 12RB
Happy to release FastPlaid, which aim to ease and accelerate ColBERT and ColPali retrieval
2025 · github.com
- 13IE
Quick note on how it works and how I've done my batch embedding engine IgniteMS. The whole thing runs as one process using Rust, reading input, tokenizing, packing batches, keeping the queue full. TensorRT handles inference. Python is only as a wrapper. I built it this way because when you use more than couple of GPUs, the GPUs stop being the problem. CPU cannot feed them fast enough. One A100 can go through batches faster than Python can tokenize and feed, so the GPU just sits there idle waiting for work. Most of my time went into optimizing this. At 8 GPUs that was basically the entire…
Jun 2026 · github.com
- 14TB
2015 · github.com
- 15FA
I was working on optimising Subversion. One of its bottleneck is computing file hashes. First thing I tried was to use OpenSSL for that. It significantly improved its performance. It made it 8x faster in some cases. However, it was not optimal to use it: - OpenSSL is heavy and bloated. It's like a whole operating system on its own. - Since OpenSSL 3.0, low-level digest APIs are deprecated, and instead they recommend to use EVP which might be slow in some cases. So I decided to create a new library for computing cryptographic digests. It basically takes OpenSSL's implementations, but with an…
Feb 2026 · github.com
- 16US
2021 · github.com
- 17TE
2021 · github.com
- 18TA
2018 · github.com
- 19RR
I built rapid-rs to eliminate the hours of boilerplate when starting a new Rust web service. One command gets you: - Auto-configured DB, logging, CORS - OpenAPI/Swagger UI at /docs - Request validation - Production-ready observability Built on Axum. Early benchmarks show ~50K req/s with 10-20MB RAM. This is v0.1 - feedback welcome! Crates: https://crates.io/crates/rapid-rs
Nov 2025 · crates.io
- 20AF
fasthex - a very optimized hex dumper with all the features other hex dumpers have too
Jun 2026 · github.com
- 21SF
Hey HN! We've just open-sourced Semble, a fast and accurate code search library built for agents. We're also releasing potion-code-16M, a small code-specialized static embedding model that powers it. Most embedding-based code search methods are either too slow to index on demand or need GPU infrastructure, while grep-style retrieval methods often cannot find the relevant content. Semble combines the speed and quality benefits of both, so agents waste less time and fewer tokens exploring. Main features: - Fast: indexes a full codebase in ~250 ms and answers queries in ~1.5 ms, all on CPU…
Apr 2026 · github.com
- 22MA
It provides zero-dependency, memory-safe operations for working with torrent files, bencode data, and magnet links.
2025 · github.com
- 23RH
Apr 2026 · github.com
- 24UI
Hey everyone! I am excited to share updates on four of my & my teams' open-source projects that take large-scale search systems to the next level: USearch, UForm, UCall, and StringZilla. These projects are designed to work seamlessly together, end-to-end—covering everything from indexing and AI to storage and networking. And yeah, they're optimized for x86 AVX2/512 and Arm NEON/SVE hardware. USearch [1]: Think of it as Meta FAISS on steroids. It's now quicker, supports clustering of any granularity, and offers multi-index lookups. Plus, it's got more native bindings than probably…
2023 · usearch-images.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →