Dev tools · alternatives · 2026

24 alternatives to Denormalized – Embeddable Stream Processing in Rust and DataFusion

tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally…

Denormalized is an embeddable stream processing engine built in Rust using Apache DataFusion. It enables incremental processing of large data volumes over time within a single application, following the efficiency… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Denormalized – Embeddable Stream Processing in Rust and DataFusion launched in 2024; newer entries below may have overtaken it.

1
Python Stream Processing with Denormalized▲20
tl;dr we’re releasing python bindings for writing stream processing jobs using the Denormalized Rust library. Hey HN, A few months ago we released Denormalized, our embeddable stream processing engine written in rust [0]. Today we’re excited to showcase python bindings for the library. Python is still very much the language of data and we wanted to make it easy for more engineers to write fast streaming jobs without having to learn Rust. Currently, you can use the library to write streaming aggregations, stream joins, and user defined aggregate functions (thought the API for this is quite…
2024 · github.com · its alternatives →
2
DuckDB for Kafka Stream Processing▲77
Hello Everyone! We built SQLFlow as a lightweight stream processing engine. We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory! DuckDB also supports a rich ecosystem of sinks and connectors! https://sql-flow.com/docs/category/tutorials/ https://github.com/turbolytics/sql-flow We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors I would love your feedback, criticisms and/or…
Dec 2025 · sql-flow.com · its alternatives →
3
A fast HNSW implementation in Rust▲111
HNSW implementation in Rust. Reference: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf - swapneel/hnsw-rust
2024 · github.com · its alternatives →
4
SwimOS Rust – A framework for real-time streaming data applications▲71
2024 · github.com · its alternatives →
5
Turbolite – a SQLite VFS serving sub-250ms cold JOIN queries from S3▲185
I built a SQLite VFS in Rust that serves cold queries directly from S3 with sub-second performance, and often much faster. It’s called turbolite. It is experimental, buggy, and may corrupt data. I would not trust it with anything important yet. I wanted to explore whether object storage has gotten fast enough to support embedded databases over cloud storage. Filesystems reward tiny random reads and in-place mutation. S3 rewards fewer requests, bigger transfers, immutable objects, and aggressively parallel operations where bandwidth is often the real constraint. This was explicitly inspired…
Mar 2026 · github.com · its alternatives →
6AF
Apache Fory Rust – 10-20x faster serialization than JSON/Protobuf▲67
Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf. Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs. Unique features that other fast serializers don't have: - Cross-language without IDL files (Rust ↔ Python/Java/Go) - Trait object serialization (Box) - Automatic circular reference handling - Schema evolution without coordination Happy to discuss design trade-offs. Benchmarks:…
Oct 2025 · fory.apache.org · its alternatives →
7
All I wanted was to grok some real-time data▲8
Hi HN, We are launching Denormalized (www.denormalized.io), a serverless real-time data platform built on Kafka and Pinot. We felt a bit burnt out by the sheer developer toil we faced when building application around the real-time data stack and set out to create a platform to allow small teams to be very productive with realtime data without having to glue together an elaborate system to serve real-time as well as time series queries. Here is our motivating post. Would appreciate any and all feedback.
2023 · teamdenormalized.substack.com · its alternatives →
8OA
Obelisk – a WASM-based deterministic workflow engine▲103
A lightweight engine for durable execution / deterministic workflows I built with Rust, wasmtime and the WASM Component Model. Its main use is running reliable, long-running workflows that can automatically resume after failures. Looking for feedback on the approach and potential use cases!
2025 · obeli.sk · its alternatives →
9
QoreDB▲95
The fast, open-source database client built with Rust
Mar 2026 · qoredb.com · its alternatives →
10
PDD – Probabilistic De-Duplication of Streams with Bloom Filters▲86
Advanced Bloom Filter Based Algorithms for Efficient Approximate Data De-Duplication in Streams - jparkie/PDD
2016 · github.com · its alternatives →
11
Tantivy▲68
A full-text, horse-speed search engine library in Rust
2022 · github.com · its alternatives →
12
Dataframes in Elixir Backed by Rust▲16
Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir - elixir-explorer/explorer
2021 · github.com · its alternatives →
13
A SIMD-accelerated noise library in Rust▲73
SIMD noise library for Rust. Contribute to verpeteren/rust-simd-noise development by creating an account on GitHub.
2018 · github.com · its alternatives →
14
Model2vec-Rs – Fast Static Text Embeddings in Rust▲60
Hey HN! We’ve just open-sourced model2vec-rs, a Rust crate for loading and running Model2Vec static embedding models with zero Python dependency. This allows you to embed text at (very) high throughput; for example, in a Rust-based microservice or CLI tool. This can be used for semantic search, retrieval, RAG, or any other text embedding usecase. Main Features: - Rust-native inference: Load any Model2Vec model from Hugging Face or your local path with StaticModel::from_pretrained(...). - Tiny footprint: The crate itself is only ~1.7 mb, with embedding models between 7 and 30 mb. Performance:…
2025 · github.com · its alternatives →
15
A local Python prototyping tool for Jupyter and Streamlit▲65
Hey HN! I built a local Python prototyping tool that is finally the Python development environment I've always wanted. It has a Jupyter notebook for data crunching, a database of your choice (Python or MongoDB), and a Streamlit app for building a frontend visualization. You can edit the Streamlit backend via an embedded VSCode editor, or locally on your own IDE. The best part for me is that the database connectors within Jupyter and Streamlit are configured out-of-the-box, so you don't need to spend time thinking about how to tie all that together - you can just pick the database you want to…
2023 · github.com · its alternatives →
16
Open-source serverless security lake powered by Rust + Apache Iceberg▲63
Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS - matanolabs/matano
2022 · github.com · its alternatives →
17
Minigraf – An embedded, bi-temporal graph database in Rust▲41
Embedded graph memory for AI agents, mobile apps, and the browser. - project-minigraf/minigraf
16d ago · github.com · its alternatives →
18
Columnar store for fast, lightweight logging▲15
Founder here, Parseable is a lightweight log ingestion and query engine written in Rust. Parseable can ingest data from existing logging agents (FluentBit, LogStash, Vector, syslog-ng and more) using HTTP + JSON output. Ingested logs are stored as semi-indexed Parquet files (on disk or S3). You can query the data with builtin query engine using SQL or use a query engines of choice like Spark, Presto, Trino and so on. We also developed a Grafana data source plugin that lets you visualise log data via Grafana. Sample dashboard link in readme. As log data grew, our industry has responded with…
2023 · github.com · its alternatives →
19
Glowstick – type level tensor shapes in stable rust▲53
Hi HN, In the past few years I've become more interested in machine learning. Since I'm sure the same is true for many here, I wanted to share this project I've been working on: glowstick uses type-directed metaprogramming to keep track of tensor shapes in Rust's type system and determine which operations are permitted or not at compile time. I find Rust has a lot of strengths when it comes to ML applications, but waiting until runtime to find shape related issues feels a bit strange since normally I don't run the code all that often while developing. Given Rust has fancy types available, I…
2025 · github.com · its alternatives →
20
LaminarDB – Streaming SQL database in Rust, zero-alloc hot path▲6
Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency - laminardb/laminardb
Feb 2026 · github.com · its alternatives →
21
Rust macro utility for batching expensive async operations▲28
rust macro util for batching expensive operations. Contribute to hackermondev/batched development by creating an account on GitHub.
2025 · github.com · its alternatives →
22
Rust framework for advanced file recognition and identification▲46
Alternative to magic.h and infer. Zero dependencies. Fully extensible. Works in no_std, async, and embedded contexts.
2025 · crates.io · its alternatives →
23
Victor, a browser-optimized vector database written in Rust▲27
Hey HN, My friend Sam and I were bored one weekend, and wanted to make something self-contained but still useful. We ended up with Victor [0], a vector database designed to run in the browser. It’s written in Rust, which gives us access to things like super-fast and compact serialization and deserialization, but it’s compiled to WASM and uses web-sys to interact with the browser’s JS APIs to write the database contents to disk. We worked hard on designing it in a way that makes sense for the browser. It uses the private origin file system [1] and stores the vectors in a compact format (one…
2023 · github.com · its alternatives →
24
Realsense-Rust – Streaming Intel RealSense from Rust▲9
High-level RealSense library in Rust
2021 · gitlab.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →