Alternatives
Products that do what Streaming JSON in Rust does
Terabytes of JSON, kilobytes of memory
- 1PJ
2018 · github.com
- 2AF
Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf. Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs. Unique features that other fast serializers don't have: - Cross-language without IDL files (Rust ↔ Python/Java/Go) - Trait object serialization (Box) - Automatic circular reference handling - Schema evolution without coordination Happy to discuss design trade-offs. Benchmarks:…
Oct 2025 · fory.apache.org
- 3SR
2024 · github.com
- 4DE
tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally…
2024 · github.com
- 5JJ
jb is a UNIX tool that creates JSON, for shell scripts or interactive use. Its "one thing" is to get shell-native data (environment variables, files, program output) to somewhere else, using JSON encapsulate it robustly. I wrote this because I wanted a robust and ergonomic way to create ad-hoc JSON data from the command line and scripts. I wanted errors to not pass silently, not coerce data types, not put secrets into argv. I wanted to leverage shell features/patterns like process substitution, environment variables, reading/streaming from files and null-terminated data. If you…
2024 · github.com
- 6
- 7AF
2021 · github.com
- 8FF
Hi HN! While working on a project that involves frequently parsing a lot of feeds, I needed a fast JavaScript-based parser to extract specific fields from feed namespaces. Existing Node packages were either too slow or merged all feed formats, losing namespace information. So I decided to write it myself and created this NPM package with a simple API. Feedsmith supports all feed formats and many popular namespaces, including: Podcast, Media, iTunes, Dublin Core, and more. It can also parse and generate OPML files. I am currently adding support for more namespaces and feed generation for RSS,…
2025 · github.com
- 9
- 10JT
2024 · github.com
- 11IJ
Built this for streaming AI tool calls. LLMs stream function arguments as JSON character-by-character. Most parsers reparse from scratch each time - O(n²) behavior that causes UI lag. This maintains parsing state, processing only new characters. True O(n) performance that stays imperceptible throughout the entire response. Ruby gem, MIT licensed. Would love feedback.
Oct 2025 · aha.io
- 12IC
2024 · github.com
- 13FA
Hello HN! I want to introduce fastgron, my new project. fastgron is a JSON to GRON converter, built to be incredibly fast – it's 40 times faster than Gron. GRON is a tool for making JSON greppable, but it can slow down with larger files. With fastgron, even a 200MB JSON file can be converted in just 1 second. Key features include streaming conversion for memory efficiency and an optimized path reconstruction for faster operations. It leverages C++ and the simdjson and fast_io libraries for speed. I welcome all feedback, suggestions, or questions. Thank you!
2023 · github.com
- 14SI
2025 · github.com
- 15JA
2020 · crates.io
- 16SF
2016 · github.com
- 17SC
Problem JSON/NDJSON is everywhere in data platforms, but compression usually breaks searchability. You either keep queryable raw stores (high I/O/egress) or compress into gz/zstd blobs (cheap to store, painful to probe). The “cloud tax” shows up as wasted reads. What I built (SEE — Semantic Entropy Encoding) A schema-aware, searchable compression codec for JSON that keeps exists/pos lookups fast while still compressing. Internals: structure-aware delta + dictionaries, a PageDir + mini-index to jump to relevant pages, and a tuned Bloom filter that skips ~99% of pages.…
Oct 2025 · github.com
- 18
- 19FF
2018 · github.com
- 20TM
I built tinykv because I kept reaching for simple persistent storage in Rust projects but found existing solutions either too complex (sled) or unmaintained (pickledb). tinykv focuses on simplicity: JSON-based, serde-powered, with optional TTL. Perfect for CLI tools, game saves, config storage. Would appreciate any feedback from the HN community!
2025 · crates.io
- 21
- 22JF
2017 · github.com
- 23PS
tl;dr we’re releasing python bindings for writing stream processing jobs using the Denormalized Rust library. Hey HN, A few months ago we released Denormalized, our embeddable stream processing engine written in rust [0]. Today we’re excited to showcase python bindings for the library. Python is still very much the language of data and we wanted to make it easy for more engineers to write fast streaming jobs without having to learn Rust. Currently, you can use the library to write streaming aggregations, stream joins, and user defined aggregate functions (thought the API for this is quite…
2024 · github.com
- 24WA
First time ever working on something like this, check it out
2024 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →