BlazeMQ – 52KB Kafka-compatible broker in C++20, zero dependencies
I built a message broker that speaks the Kafka wire protocol, so any Kafka client (librdkafka, kafka-python, kcat, etc.) works without code changes. The entire binary is 52KB. No JVM, no ZooKeeper, no third-party libraries — just C++20 with kqueue/epoll. Starts in <10ms, uses 0% CPU when idle. I built this because running Kafka locally for development is painful — gigabytes of RAM, slow startup, ZooKeeper/KRaft configuration. I just wanted something that accepts produce requests and gets out of the way. Technical details: - Single-threaded event loop (kqueue on macOS, epoll on…
What it does
In the maker’s words, at launch
I built a message broker that speaks the Kafka wire protocol, so any Kafka client (librdkafka, kafka-python, kcat, etc.) works without code changes. The entire binary is 52KB. No JVM, no ZooKeeper, no third-party libraries — just C++20 with kqueue/epoll. Starts in <10ms, uses 0% CPU when idle. I built this because running Kafka locally for development is painful — gigabytes of RAM, slow startup, ZooKeeper/KRaft configuration. I just wanted something that accepts produce requests and gets out of the way. Technical details: - Single-threaded event loop (kqueue on macOS, epoll on Linux) - Memory-mapped log segments (1GB pre-allocated, sequential I/O) - Lock-free SPSC/MPSC ring buffers with cache-line alignment - Kafka protocol v0-v3 including flexible versions (ApiVersions, Metadata, Produce) - Auto-topic creation on first produce or metadata request The most interesting bug I hit: librdkafka sends ApiVersions v3, which uses Kafka's "flexible versions" encoding. But there's a special exception in the protocol — ApiVersions responses must NOT include header tagged_fields for backwards compatibility. One extra byte shifted every subsequent field, causing librdkafka to compute a ~34GB malloc that crashed immediately. Current limitations: no consumer groups, no replication, single-threaded, no auth. It's v0.1.0 — consume support is next. MIT licensed, runs on macOS (Apple Silicon + Intel) and Linux.
Does the same job
all alternatives →- KSKQ – Simple Job Queue for Python Using Kafka2016 · github.com · ▲116
- FAFranz – A desktop client for Apache Kafka2024 · github.com · ▲45
This is the source code to Franz, a native desktop client for Apache Kafka I've been working on for the past ~three years or so. Around 2021, I started using Kafka at a startup and I didn't like any of the clients that were available at the time, so I decided to build my own in my spare time[1]. I wanted a solid native experience on the Mac that wasn't web based and didn't require any Docker containers to run. A few months ago, I decided to make the code source-available[2] in part because I like the idea of my users being able to see the code they're running and in part because I want to…
- AAAutoMQ - A Cost-Effective Kafka Distro That Can Autoscale in Seconds2024 · github.com · ▲58
- K0Kafka 0.8.0 on Cloudflare Workers2022 · github.com · ▲148
- KAKowl – An Open Source WebUI for Apache Kafka with a good UI and UX2020 · github.com · ▲164
- PDPicoMQ – Durable Streams over HTTP, on object storage13d ago · picomq.com · ▲159
PicoMQ is a Rust server for Durable Streams, built on Object Store. Cheap, URL-addressable, granular streams (create/append/read/long-poll/SSE), with Pico Protocol or Durable Streams Protocol as the facade. S3Stream is the stream storage primitive, used in AutoMQ, shipped as a Rust library. Coordination is a command log in Postgres.
More dev tools this month
the category →
The first open-source price index for GPU compute
Dev tools · 10d ago · getcomputable.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
