Dev tools · alternatives · 2026

24 alternatives to Sparse Matrix-Vector Multiplication that works at 30–90% sparsity

To get benefits from sparsity, you usually need to have very sparse matrices, impose some structure on the sparsity pattern or have specialized hardware. None of it is the case if you want to rune pruned LLMs on consumer devices. I wanted to see how far can you push it on a GPU and ended up with this. Blog: https://www.grizzlytech.dev/blog/macko-spmv Paper: https://arxiv.org/abs/2511.13061 Code (example with torch): https://github.com/vlejd/macko_spmv

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.

1
Speeding up LLM inference 2x times (possibly)▲419
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
2
I've built a locally running Perplexity clonefree▲669
The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
2024 · github.com · its alternatives →
3
Matrix Multiplication with Half the Multiplications▲310
Algebraic enhancements for GEMM & AI accelerators. Contribute to trevorpogue/algebraic-nnhw development by creating an account on GitHub.
2024 · github.com · its alternatives →
4
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning▲385
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
5
How I topped the HuggingFace open LLM leaderboard on two gaming GPUs▲495
I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1. As of 2026, the top 4 models on that leaderboard are still descendants. The weird finding: single-layer duplication does nothing. Too few layers, nothing. Too many, it gets worse. Only circuit-sized blocks of ~7 layers work. This suggests pretraining carves out discrete functional circuits in the layer stack that only work when preserved whole. The whole thing was developed on 2x RTX 4090s in my basement. I'm…
Mar 2026 · dnhkng.github.io · its alternatives →
6
Run TRELLIS.2 Image-to-3D generation natively on Apple Silicon▲202
I ported Microsoft's TRELLIS.2 (4B parameter image-to-3D model) to run on Apple Silicon via PyTorch MPS. The original requires CUDA with flash_attn, nvdiffrast, and custom sparse convolution kernels: none of which work on Mac. I replaced the CUDA-specific ops with pure-PyTorch alternatives: a gather-scatter sparse 3D convolution, SDPA attention for sparse transformers, and a Python-based mesh extraction replacing CUDA hashmap operations. Total changes are a few hundred lines across 9 files. Generates ~400K vertex meshes from single photos in about 3.5 minutes on M4 Pro (24GB). Not as fast as…
Apr 2026 · github.com · its alternatives →
7
Pytorch_block_sparse: smaller, faster neural networks▲6
Today I am excited to release pytorch-block-sparse: a *drop-in* replacement of @PyTorch Linear with GPU-efficient sparsity: 75% sparsity➡️4x less memory + 2x the speed! Code & tutorials to use to train your own sparse models: https://t.co/HMO11zWAjk
2020 · twitter.com · its alternatives →
8
Graph Processing with Postgres and GraphBLAS▲106
Accelerated Sparse Linear Algebra with Postgres and SuiteSparse - OneSparse/OneSparse
2019 · github.com · its alternatives →
9
I built a tensor library from scratch in C++/CUDA▲119
Hi HN, Over the past few months, I've been building `dsc`, a tensor library from scratch in C++/CUDA. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally. The key features are: - C++ core with CUDA support written from scratch. - A familiar, PyTorch-like Python API. - Runs real models: it's complete enough to load a model like Qwen from HuggingFace and run inference on both CUDA and CPU with a single line change[1]. - Simple, built-in observability for both Python and C++. Next on the roadmap is…
2025 · github.com · its alternatives →
10WM
We made glhf.chat – run almost any open-source LLM, including 405B▲161
Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…
2024 · glhf.chat · its alternatives →
11
Luminal – Open-source, search-based GPU compiler▲153
Hi HN, I’m Joe. My friends Matthew, Jake and I are building Luminal (https://luminalai.com/), a GPU compiler for automatically generating fast GPU kernels for AI models. It uses search-based compilation to achieve high performance. We take high level model code, like you'd have in PyTorch, and generate very fast GPU code. We do that without using LLMs or AI - rather, we pose it as a search problem. Our compiler builds a search space, generates millions of possible kernels, and then searches through it to minimize runtime. You can try out a demo in `demos/matmul` on mac to…
2025 · github.com · its alternatives →
12
High-Performance .NET Bindings for the Vello Sparse Strips CPU Renderer▲18
Vello Sparse Strips .NET Bindings. Contribute to wieslawsoltes/SparseStrips development by creating an account on GitHub.
Nov 2025 · github.com · its alternatives →
13
TurboQuant for vector search – 2-4 bit compression▲89
A vector index built on TurboQuant, written in Rust with Python bindings - RyanCodrai/turbovec
Mar 2026 · github.com · its alternatives →
14SA
Sparser – A Multilanguage Parser▲65
* https://sparser.io * https://github.com/Unibeautify/sparser This is my attempt at creating a universal language parser. It attempts to solve a couple of problems: * Support multiple languages * Recursively extend support to languages embedded within other languages * Output a uniform format for all supported languages This is a personal project so any feedback would be helpful. Something interesting I found after I built it is that this parser is not as fast to write output as many other JavaScript parsers, but its output is much faster to read from due to the…
2019 · its alternatives →
15
MiniPCs.zip – Charting the Pareto frontier of Mini PCs▲115
The overall idea is to chart out the thousands of Mini PCs by benchmark and reveal the Pareto Front so you can get the most Compute per Dollar. Definitely a labor of love as I have a number of Mini PCs for my "homelab" (TrueNAS, piHole, Plex, basic stuff). It uses Gemini to extract specs from listings (since they're not often strongly categorized). Quick blog post here: https://luke.zip/posts/pareto-pcs/
Jun 2026 · minipcs.zip · its alternatives →
16
OneSparse – Open Source Accelerated Sparse Linear Algebra for Postgres▲13
Accelerated Sparse Linear Algebra with Postgres and SuiteSparse - OneSparse/OneSparse
2024 · github.com · its alternatives →
17
Local fine tuning for Mistral and SDXL, GPU mem/latency optimization▲36
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM. It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization It's a pretty…
2023 · docs.helix.ml · its alternatives →
18
Portable matrix multiplication library▲15
Portable matrix multiplication library in C. Contribute to webDva/matrixmul development by creating an account on GitHub.
2018 · github.com · its alternatives →
19AG
A GPU-accelerated binary vector index▲65
This is a vector index I built that supports insertion and k-nearest neighbors (k-NN) querying, optimized for GPUs. It operates entirely in CUDA and can process queries on half a billion vectors in under 200 milliseconds. The codebase is structured as a standalone library with an HTTP API for remote access. It’s intended for high-performance search tasks—think similarity search, AI model retrieval, or reinforcement learning replay buffers. The codebase is located at https://github.com/rodlaf/BinaryGPUIndex.
2025 · rlafuente.com · its alternatives →
20
Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO)▲21
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com · its alternatives →
21
A reimplementation of Google's sparsehash in C▲22
A simple re-implementation of Google's sparsehash as a learning excercise. - qpfiffer/Simple-Sparsehash
2015 · github.com · its alternatives →
22
TiGrIS, a tiling compiler that fits ML models onto embedded devices▲20
Ahead-of-time compiler that fits ML models on embedded devices with hard memory budgets. - raws-labs/tigris
Apr 2026 · github.com · its alternatives →
23
20x faster pgvector alternative written in Rust▲23
Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database. - tensorchord/pgvecto.rs
2023 · github.com · its alternatives →
24
LeetGPU – LeetCode for GPU Programming▲9
After the incredible response to our launch of the first online CUDA playground, we have just shipped something we think all you GPU programming and ML enthusiasts will love. Introducing LeetGPU Challenges--the place to compete on writing the fastest CUDA kernels. We have problems like matrix multiplication, agent simulation, multi-head self-attention, with more dropping every couple of days! We have a lot of really cool things coming up, including support for PyTorch, TensorFlow, JAX, TinyGrad; Multi-GPU programs; H100, V100, A100 GPU options Give it a try and let us know what you think!
2025 · leetgpu.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →