Alternatives
Products that do what StreamHouse – S3-native Kafka alternative written in Rust does
Hey HN, I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost. How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box. What's there today: - Producer API with batching, LZ4 compression, and…
- 1SS
May 2026 · github.com
- 2GO
Hi HN! We are Ashish and Armend, founders of GlassFlow. We just launched our open-source streaming ETL that deduplicates and joins Kafka streams before ingesting them to ClickHouse https://github.com/glassflow/clickhouse-etl Why we built this: Dedup with batch data is straightforward. You load the data into a temporary table. Then, find only the latest versions of the record through hashes or keys and keep them. After that, move the clean data into your main table. But have you tried this with streaming data? Users of our prev product were running real-time analytics…
2025 · github.com
- 3SL
S2 was on HN for our intro blog post a year ago (https://news.ycombinator.com/item?id=42480105). S2 started out as a serverless API — think S3, but for streams. The idea of streams as a cloud storage primitive resonated with a lot of folks, but not having an open source option was a sticking point for adoption – especially from projects that were themselves open source! So we decided to build it: https://github.com/s2-streamstore/s2 s2-lite is MIT-licensed, written in Rust, and uses SlateDB (https://slatedb.io) as its storage engine. SlateDB is an…
Jan 2026 · github.com
- 4PD
PicoMQ is a Rust server for Durable Streams, built on Object Store. Cheap, URL-addressable, granular streams (create/append/read/long-poll/SSE), with Pico Protocol or Durable Streams Protocol as the facade. S3Stream is the stream storage primitive, used in AutoMQ, shipped as a Rust library. Coordination is a command log in Postgres.
13d ago · picomq.com
- 5
- 6WA
Dec 2025 · github.com
- 7DF
Hello Everyone! We built SQLFlow as a lightweight stream processing engine. We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory! DuckDB also supports a rich ecosystem of sinks and connectors! https://sql-flow.com/docs/category/tutorials/ https://github.com/turbolytics/sql-flow We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors I would love your feedback, criticisms and/or…
Dec 2025 · sql-flow.com
- 8PS
Hello HN, I am Sai Srirampur, one of the Co-founders of PeerDB. (https://github.com/PeerDB-io/peerdb). We spent the past 7 months building a solid experience to replicate data from Postgres to data warehouses. Now we're expanding to queues. PeerDB Streams provides a simple and native way to replicate changes as they happen in Postgres to Queues (Kafka, Redpanda, Google PubSub, etc). We use Postgres logical decoding to enable Change Data Capture (CDC). Blog post here: https://blog.peerdb.io/peerdb-streams-simple-native-postgres.... 10-min quickstart here:…
2024
- 9FA
This is the source code to Franz, a native desktop client for Apache Kafka I've been working on for the past ~three years or so. Around 2021, I started using Kafka at a startup and I didn't like any of the clients that were available at the time, so I decided to build my own in my spare time[1]. I wanted a solid native experience on the Mac that wasn't web based and didn't require any Docker containers to run. A few months ago, I decided to make the code source-available[2] in part because I like the idea of my users being able to see the code they're running and in part because I want to…
2024 · github.com
- 10SP
This is a MVP for Serverless Postgres. 1/ It uses Fly.io[0], which can automatically pause your database after all connections are released (and start it again when new connections join). 2/ It uses Oriole[1], a Postgres extension with experimental support for S3 / Decoupled Storage[2]. 3/ It uses Tigris[3], Globally Distributed S3-Compatible Object Storage. Oriole will automatically backup the data to Tigris using background workers. I wouldn't recommend using this in production, but I think it's in a good spot to provoke some discussion and ideas. You can get it running…
2024 · github.com
- 11
- 12
When building Hookie, I worked with NATS Jetstream a lot. While you can do everything using the CLI, I prefer the feeling of a nice GUI better. So I built one.
23d ago · streambench.app
- 13IB
I created a 100% Open source Company-wide Self-hosted File Sharing Solution for Teams Recently, I wanted to share HD images and video files with my graphic designer. She’s exceptional at her craft but isn’t familiar with AWS S3 So, I got an idea and built this. Github Repo: https://github.com/rohitg00/s3-file-share-for-free Detailed Guide: https://ghumare64.medium.com/i-built-a-company-wide-self-hos...
2024 · s3-file-share-for-free-35n2u.kinsta.app
- 14GI
Over the past few years I’ve been thinking about how I could build SaaS and deploy it on my own infrastructure without needing to use any cloud platforms like AWS or GCP. In this repo I document my progress on building a clone of AWS S3 that functions the same as S3 using an exclusively open-source technology stack.
2024 · github.com
- 15GD
2017 · github.com
- 16TA
I built a SQLite VFS in Rust that serves cold queries directly from S3 with sub-second performance, and often much faster. It’s called turbolite. It is experimental, buggy, and may corrupt data. I would not trust it with anything important yet. I wanted to explore whether object storage has gotten fast enough to support embedded databases over cloud storage. Filesystems reward tiny random reads and in-place mutation. S3 rewards fewer requests, bigger transfers, immutable objects, and aggressively parallel operations where bandwidth is often the real constraint. This was explicitly inspired…
Mar 2026 · github.com
- 17SC
Dec 2025 · github.com
- 18A1
AutoMQ is a fully open-source, next-generation Kafka built on top of S3. Version 1.5.0 is a very important release. In March of this year, Confluent officially launched the commercial capability of Tableflow. Now, with the AutoMQ open-source software, you can also experience this killer feature. Simply put, for Kafka streaming data, AutoMQ can automatically store it in S3 in Iceberg Table format, so you no longer need to manage Flink Jobs and Spark Jobs yourself to perform ETL operations and convert Kafka data into table format. We believe this will be the new paradigm for Kafka stream data…
2025 · github.com
- 19CS
We started CallFS after yet another late-night “why did the uploads vanish?” incident. Our small team had stitched together rsync, a fragile NFS mount, and an S3 bucket—none of it observable, all of it waiting to bite us. So we wrote a single-process file service in Go that: • Speaks the S3 API (so existing tooling works). • Stores hot data on local disks for speed; cold data can sit in any S3-compatible bucket. • Exposes Prometheus metrics and JSON logs by default, because “what happened?” shouldn’t be guesswork. • Ships as a ~25 MB static binary—no external deps, MIT license. Today it’s…
2025 · github.com
- 20AS
Hey everyone, I’m Hunter, the founder of Archil. Archil is transforming object storage, like Amazon S3, into infinite, local file systems that provide instant access to massive data sets. Last year, we launched Archil’s NFS-based product publicly on Hacker News (https://news.ycombinator.com/item?id=42174204), and we were absolutely thrilled to see the response of this community. Since our last launch, we took a 10 month company-wide bet to build our own, custom storage protocol to deliver true, local-like performance to cloud instances – by behaving closer to a block storage…
Sep 2025
- 21AA
2024 · github.com
- 22DS
Hey, I'm a co-founder at ElectricSQL. Durable Streams is the delivery protocol underneath our Postgres sync engine—we've been refining it in production for 18 months. The core idea: streams get their own URL and use opaque, monotonic offsets. Clients persist the last offset they processed and resume with "give me everything after X." No server-side session state, CDN-cacheable, plain HTTP. We kept seeing teams reinvent this for AI token streaming and real-time apps, so we're standardizing it as a standalone protocol. The repo has a reference Node.js server and TypeScript client. Would love…
Dec 2025 · github.com
- 23
- 24OS
2024 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →