Streaming JSON in Rust
Terabytes of JSON, kilobytes of memory
What it does
Streaming JSON is a set of Rust libraries for parsing and transforming JSON using a sliding window approach, without loading the full document into memory. It includes: - "RJiter": a streaming JSON parser - "scan_json": trigger actions during parsing by matching keys and nested structures - "id_transform": copy input to output, a base for JSON rewriting - "ddb_convert": convert JSON to/from DynamoDB format, 12× faster than Python and 2× faster than a baseline Rust implementation
Does a similar job
all alternatives →- PJPython JSON library in Rust, faster than ujson2018 · github.com · ▲40
- AFApache Fory Rust – 10-20x faster serialization than JSON/ProtobufOct 2025 · fory.apache.org · ▲67
Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf. Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs. Unique features that other fast serializers don't have: - Cross-language without IDL files (Rust ↔ Python/Java/Go) - Trait object serialization (Box) - Automatic circular reference handling - Schema evolution without coordination Happy to discuss design trade-offs. Benchmarks:…
- SRSwimOS Rust – A framework for real-time streaming data applications2024 · github.com · ▲71
- DEDenormalized – Embeddable Stream Processing in Rust and DataFusion2024 · github.com · ▲125
tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally…
- JTJSON-Threat-Protection Rust High-Performance Crate2024 · github.com · ▲34
- IJIncremental JSON parser for streaming LLM tool calls in RubyOct 2025 · aha.io · ▲13
Built this for streaming AI tool calls. LLMs stream function arguments as JSON character-by-character. Most parsers reparse from scratch each time - O(n²) behavior that causes UI lag. This maintains parsing state, processing only new characters. True O(n) performance that stays imperceptible throughout the entire response. Ruby gem, MIT licensed. Would love feedback.
More dev tools this month
the category →
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Open-source GTM skills for technical founders
Dev tools · 30d ago · gtmcofounder.com
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
