We Beat MLPerf: Modern Storage for KV Offload and LLM Training
High performance storage engine for efficient LLM inference and GPU Training.
We Beat MLPerf: Modern Storage for KV Offload and LLM Training launched on September 5, 2026 with 35 votes, #716 of 8,093 launches that month and more than 95% of that year's launches.
In plain words
OpenLake's Infinity Core I/O Engine is a storage system designed for large language model inference and GPU training workloads. It delivers high-speed read and write performance for checkpoint operations, achieving 11.55 GiB/s reads and 6.72 GiB/s writes in MLPerf Storage v3.0 benchmarks. The product targets organizations running LLM training and inference operations that require efficient data handling at scale.
written from the facts on this page · September 2026
From the sources
OpenLake’s Infinity Core I/O Engine delivered 6.72 GiB/s writes and 11.55 GiB/s reads in the MLPerf Storage v3.0 Llama 3.1 8B checkpointing benchmark. September 1, 2026. MLCommons released the MLPerf Storage v3.0 results . MLPerf Storage is an industry standard benchmark for evaluating storage systems under representative AI training and inference workloads. OpenLake achieved the highest read and write bandwidth among the published Closed division S3 results for Llama 3.1 8B checkpointing with one client node and one data parallel instance. These five submissions represent the closest like for like comparison in the published results, although the underlying storage configurations differ.…from theopenlake.com
Does a similar job
all alternatives →
Cuts Long Horizon Inference Costs by 50% via external KV Cache OffloadJul 2026 · github.com · ▲22Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
TensorZero2025 · github.com · ▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
- WMWe made glhf.chat – run almost any open-source LLM, including 405B2024 · glhf.chat · ▲161
Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…
ZSE – Open-source LLM inference engine with 3.9s cold startsFeb 2026 · github.com · ▲58I've been building ZSE (Z Server Engine) for the past few weeks — an open-source LLM inference engine focused on two things nobody has fully solved together: memory efficiency and fast cold starts. The problem I was trying to solve: Running a 32B model normally requires ~64 GB VRAM. Most developers don't have that. And even when quantization helps with memory, cold starts with bitsandbytes NF4 take 2+ minutes on first load and 45–120 seconds on warm restarts — which kills serverless and autoscaling use cases. What ZSE does differently: Fits 32B in 19.3 GB VRAM (70% reduction vs FP16) — runs…
Open-source real time data framework for LLM applications2024 · getindexify.ai · ▲92Hey HN, I am the founder of Tensorlake. Prototyping LLM applications have become a lot easier, building decision making LLM applications that work on constantly updating data is still very challenging in production settings. The systems engineering problems that we have seen people face are - 1. Reliably process ingested content in real time if the application is sensitive to freshness of information. 2. Being able to bring in any kind of model, and run different parts of the pipeline on GPUs and CPUs. 3. Fault Tolerance to ingestion spike, compute infrastructure failure. 4. Scaling compute,…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 23d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 24d ago · fastrouter.ai
Launched alongside, September 2026
the whole month →
E-ink bird frame for Raspberry Pi - real-time bird detection by audio, fully local AI, rendered as real, hand-cut 1800s bird illustrations. - arnegiacomo/fugleramme
AI · Sep 2026 · github.com
- WI
Special relativity simulator in which light crawls along at a human speed, so Lorentz contraction, aberration and Doppler shift happen at walking pace.
Life & fun · Sep 2026 · rivendell.dmitrybrant.com
Start coding agents, control sessions, review PRs. Anywhere.
Dev tools · Sep 2026 · kilo.ai
