An RDMA/Infiniband Distributed Cache for Fast Inference and Training
A high-performance RDMA distributed file system for fast LLM Inference and GPU Training. - GitHub - blackbird-io/blackbird: A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.
An RDMA/Infiniband Distributed Cache for Fast Inference and Training launched on September 19, 2025 with 13 votes, #826 of 8,160 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
This is a distributed cache system designed for machine learning applications, built on RDMA and Infiniband technology. It serves developers and researchers who need fast data access during model inference and training workloads. The system leverages high-speed networking protocols to reduce latency in data retrieval, making it particularly useful for large-scale machine learning operations where data transfer speed is a bottleneck.
written from the facts on this page · September 2026
From the sources
Blackbird draws inspiration from Microsoft/FARM and RDMA based KV store among other projects. It also takes cues from Redis for its simplicity and ubiquity. It delivers intelligent data placement, enabling applications to seamlessly offload data to a high-performance tiered system managing placement and latency for you. Use cases: HPC & ML training/inference pipelines, realtime analytics, feature stores, and metadata-heavy services where Redis/Memcached lack tiering or RDMA, and Alluxio is too heavyweight. # 1) Start etcd etcd --listen-client-urls http://localhost:2379 \ --advertise-client-urls http://localhost:2379 # 2) Start Keystone (example) ./examples/keystone_example --etcd-endpoints…from github.com
In the maker’s words, at launch
Check it out!
Does a similar job
all alternatives →
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
General ComputeMay 2026 · generalcompute.com · ▲315AI models that run on an inference cloud optimized for speed
Cuts Long Horizon Inference Costs by 50% via external KV Cache OffloadJul 2026 · github.com · ▲22Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…
We Beat MLPerf: Modern Storage for KV Offload and LLM TrainingSep 2026 · theopenlake.com · ▲35High performance storage engine for efficient LLM inference and GPU Training.
Initial release of the distributed TensorFlow runtime2016 · github.com · ▲85An Open Source Machine Learning Framework for Everyone - tensorflow/tensorflow/core/distributed_runtime at master · tensorflow/tensorflow
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, September 2025
the whole month →
The site hit #1 on HN about a year ago. Got enough traffic that I turned it into a proper API. What's there: - REST API — 35k products, manually filterable by category, free key, 60 req/min - CLI — npx anycrap random -c food - faker.js plugin — bundled, no API call, works offline - HuggingFace dataset — full 35k rows for ML/training https://anycrap.shop/developers
AI · 2025 · anycrap.shop
I made a built-from scratch Wayland Compositor to display any GUI app* in the terminal! I think there is a lot of unexplored potential in custom Wayland compositors, a lot of really cool things you can embed existing applications into! So, I started with embedding apps into the terminal because that is the easiest input/output (output is just utf-8 and I use the great `chafa` library for that, and I just read from stdin for the input). If you have any other ideas for cool Wayland compositors, let me know. I purposedly wrote 80% the app in Typescript to appeal to the most developers and…
Dev tools · 2025 · github.com
- IR
Years ago I stumbled across a basic version of this concept and it stuck with me. I knew if I was ever going to take on such a project, it would need to be flawless, but without coding experience it was just another idea that would never happen. By the end of 2024, as AI coding tools exploded everywhere, I finally had a way to make it real. I started from zero knowledge and spent months collaborating with AI agents as a learning experience. Every pixel and every function went through me. The AI translated what I asked for into code, but every decision was human. I didn't use existing OS…
AI · 2025 · mitchivin.com
