Alternatives
Products that do what A new engine to run Kimi K3 on a laptop does
Kimi K3 has 2.78 trillion parameters and ships as 1.42 TB of weights. It clearly does not fit in the memory of a laptop. But K3 is a Mixture-of-Experts model. For each token, only a small fraction of its 896 experts per layer is activated. That changes the problem: the entire model does not need to be resident in RAM, as long as the weights required by each token can be reached quickly enough. We built WASTE — the Weight-Aware Streaming Tensor Engine — to explore that idea. WASTE keeps the dense, repeatedly used part of the model resident in memory, stores the routed experts in an…
- 1
- 2
- 3
Run the full 2.78-trillion-parameter Kimi K3 model or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine. - sqliteai/warp
Jul 2026 · github.com
- 4MO
I wanted to share our new speech to text model, and the library to use them effectively. We're a small startup (six people, sub-$100k monthly GPU budget) so I'm proud of the work the team has done to create streaming STT models with lower word-error rates than OpenAI's largest Whisper model. Admittedly Large v3 is a couple of years old, but we're near the top the HF OpenASR leaderboard, even up against Nvidia's Parakeet family. Anyway, I'd love to get feedback on the models and software, and hear about what people might build with it.
Feb 2026 · github.com
- 5
- 6
- 7D3
I replicated David Ng's RYS method (https://dnhkng.github.io/posts/rys/) on consumer AMD GPUs (RX 7900 XT + RX 6950 XT) and found something I didn't expect. Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act as indivisible cognitive units. Duplicate the right block and the model runs its reasoning pipeline twice. No weights change. No training. The model just thinks longer. The results on standard benchmarks (lm-evaluation-harness, n=50): Devstral-24B, layers 12-14 duplicated once: - BBH Logical Deduction: 0.22 → 0.76…
Mar 2026 · github.com
- 8
- 9
We recently used DeepSeek V4 Flash as a teacher for finance tasks with GPT-OSS-120B. Distillation works well on this problem. At a constrained 8k token budget, our self-distilled 120B scores 83.61% on FinanceReasoning, above Kimi K3 (81.93%) and Inkling (65.13%). We released the 20B open weights. With V4 as the teacher though, we realized it would be timely to measure if the censorship characteristic of it transferred to the distilled version of the base model. tl;dr it didn't, the teacher answered politically sensitive questions 7 SDs differently than expected, but the distilled model's…
Jul 2026 · ctgt.ai
- 10
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup
Aug 2026 · github.com
- 11AT
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
28d ago · mikeayles.com
- 12EL
2023 · github.com
- 13OP
Open Prompts is the dataset used to build krea.ai. The data comes from the Stability AI Discord and includes around 10M images from 2M prompts. You can use it for creating semantic search engines of prompts, training LLMs, fine-tuning image-to-text models like BLIP, or extracting insights from the data—like the most common combinations of modifiers.
2022 · github.com
- 14IR
The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
Mar 2026 · github.com
- 15IB
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…
2025 · tinytpu.com
- 16
- 17LT
hey guys. the other day i was migrating hosting providers and i just needed something not too heavy and convenient to spin up my backups for awhile and realised there is almost nothing out there. kimchi hasn't been updated for years and cockpit is heavy. so here's something i came up with in a couple hours because of a sudden urge, nothing fancy just basic creation with cloud init, lifecycle management and image/storage, but it's modern-ish and it compiles to a 8.4mb binary inclusive of the embedded web UI, CLI and API, and only dep is libvirt.
Sep 2025 · github.com
- 18RG
I wanted to know how fast a 26B mixture-of-experts model could run on a desktop CPU with no GPU. Got ~40 tok/s single-stream (lossless) and ~124 batched. The surprising part was the byte budget: for this model you compress the output head (32% of per-token bytes), not the experts (16%). The writeup has the bandwidth roofline and the dead-ends; the repo has the reproducible recipe. Happy to answer questions. Repo: https://github.com/arun-prasath2005/gemma4-cpu-moe
Jun 2026 · apeg.dev
- 19
Independent from-scratch implementation of the Kimi K3 architecture (arXiv:2607.24653v1): KDA, NoPE, latent-space MoE, and the systems co-designs. Table 1 reproduced to 0.09%. - TimRots/kimi3
Aug 2026 · github.com
- 20RX
Hi all, long time lurker, first time poster. I want to share with you all something we've been working on for a while at Lambda: the Razer x Lambda Tensorbook: https://www.youtube.com/watch?v=wMh6Dhq7P_Q But before I tell you about it, I want to make this all about me, because I built this for me. See, while I'm genuinely interested in hearing from the community what you think as this is the culmination of a lot of effort from a lot of people across so many different fields (seriously, the number of folks across manufacturing, engineering, design, logistics, and marketing who…
2022
- 21
High performance storage engine for efficient LLM inference and GPU Training.
1d ago · theopenlake.com
- 22
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com
- 23I4
It's our new text-to-image model: a 9.3B single-stream diffusion transformer trained entirely from scratch. We focused heavily on controllability through structured JSON prompts, with strong text rendering, spatial awareness through bounding box guidance, and color palette control. It has the best text rendering of any open-weight model we've tested so far, and the NF4 quantized checkpoint runs on a single 24GB GPU. For more technical details and examples see our blog post: https://ideogram.ai/blog/ideogram-4.0/ We will be happy to answer any questions :)
Jun 2026 · github.com
- 24
Local, gradient-free neuro-symbolic memory engine combining Hyperdimensional Computing (HDC/VSA), Hebbian plasticity, and graph triples for offline AI. - roandejager/Hillock
7d ago · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →