StreamHouse – S3-native Kafka alternative written in Rust
Hey HN, I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost. How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box. What's there today: - Producer API with batching, LZ4 compression, and…
In plain words
StreamHouse is an open-source streaming platform built in Rust that replaces Kafka's broker storage with direct writes to S3. It maintains Kafka-compatible semantics while reducing operational overhead by eliminating broker disk management and replication tuning. A stateless server handles partition routing and metadata storage, while producers batch and compress records directly to S3, and consumers read from S3 with local caching. It supports producer and consumer APIs, Kafka-compatible protocol, REST and gRPC interfaces, and achieves throughput of 30K+ records per second.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN, I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost. How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box. What's there today: - Producer API with batching, LZ4 compression, and offset tracking (62K records/sec) - Consumer API with consumer groups, auto-commit, and multi-partition fanout (30K+ records/sec) - Kafka-compatible protocol (works with existing Kafka clients) - REST API, gRPC API, CLI, and a web UI - Docker Compose setup for trying it locally in 5 minutes The cost model is what motivated this. Kafka's storage costs scale with replication factor × retention × volume. With S3 at $0.023/GB/month, storing a TB of events costs ~$23/month instead of hundreds on broker EBS volumes. Written in Rust, ~50K lines across 15 crates. Apache 2.0 licensed. GitHub: https://github.com/gbram1/streamhouse Happy to answer questions about the architecture, tradeoffs, or what I learned building this.
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
