UL-SMF – Open-source linear-complexity ~300x KV-cache compression
The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 3
UL-SMF – Open-source linear-complexity ~300x KV-cache compression launched on August 17, 2026 with 13 votes, #880 of 23,274 launches that month and more than 90% of that year's launches.
In plain words
UL-SMF is an open-source hardware-software compression system designed for developers working with long-context Transformer models. It reduces Key-Value cache memory requirements by combining Finite Scalar Quantization with 16-dimensional latent mapping, achieving up to 384x compression while retaining over 94% semantic accuracy. The tool addresses memory bottlenecks in inference workloads, making it relevant for teams deploying large language models with extended context windows.
written from the facts on this page · September 2026
From the sources
The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 384x while maintaining >94% semantic retention. - liventruth/UL-SMF-Cache-Compression
Does a similar job
all alternatives →
Needle2: 14MB agentic LLM for phones, wearables, smart home and robotsAug 2026 · cactuscompute.com · ▲537Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
- ATA tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo)Aug 2026 · mikeayles.com · ▲79
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
We Beat MLPerf: Modern Storage for KV Offload and LLM TrainingSep 2026 · theopenlake.com · ▲35High performance storage engine for efficient LLM inference and GPU Training.
- OAOverfitted a 900KB Transformer to Compress a 100MB CSV into 7MBJun 2026 · ▲112
I built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files. Instead of training the model to generalize, I train a 900KB transformer to memorize a single file and predict the next byte. Those predictions are fed into an arithmetic coder to produce the compressed output. On a 100MB NYC taxi CSV, it compresses to about 7MB (~0.5 bits/byte). On a 100MB slice of enwik9, it compresses to about 21MB (~1.68 bits/byte). It's pretty slow right now (roughly 20–30 minutes of training and 45 minutes each for compression and decompression on my…
A general-purpose encoder-decoder framework for Tensorflow2017 · github.com · ▲54A general-purpose encoder-decoder framework for Tensorflow - google/seq2seq
I ran a language model on a PS2Mar 2026 · github.com · ▲46The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
More dev tools this month
the category →
Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.
Dev tools · 1d ago · github.com
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 29d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 27d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 24d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 22d ago · talanoa.email
Source: Show HN post ↗
Launched alongside, August 2026
the whole month →- TL
Life & fun · Aug 2026 · louisabraham.github.io
Clipto MCP▲655Hey HN, We recently built Clipto. It’s a tool that lets you search over terabytes of video, audio, and images on your computer, without relying on the cloud. Motivation: we probably all had this similar experience, we know a moment exists in a video or audio, but finding it takes hours scrubbing the timeline. You can send all the media to process in the cloud, but it’s slow, expensive and raises privacy concerns. So we decided to build our own on-device media search engine. How it works (high level): 1. We ingest video, audio, image; normalize formats via ffmpeg; run content analysis to…
Work · Aug 2026 · clipto.com
Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…
Life & fun · Aug 2026 · toneyalexander.github.io
Astute▲606Automate your B2B brand going viral, with new media creators
AI · Aug 2026 · company-app.joinastute.com