Alternatives
Products that do what Sparse Matrix-Vector Multiplication that works at 30–90% sparsity does
To get benefits from sparsity, you usually need to have very sparse matrices, impose some structure on the sparsity pattern or have specialized hardware. None of it is the case if you want to rune pruned LLMs on consumer devices. I wanted to see how far can you push it on a GPU and ended up with this. Blog: https://www.grizzlytech.dev/blog/macko-spmv Paper: https://arxiv.org/abs/2511.13061 Code (example with torch): https://github.com/vlejd/macko_spmv
- 1US
2015 · peterhrynkow.com
- 2VA
2013 · zhehaomao.com
- 3
- 4SA
* https://sparser.io * https://github.com/Unibeautify/sparser This is my attempt at creating a universal language parser. It attempts to solve a couple of problems: * Support multiple languages * Recursively extend support to languages embedded within other languages * Output a uniform format for all supported languages This is a personal project so any feedback would be helpful. Something interesting I found after I built it is that this parser is not as fast to write output as many other JavaScript parsers, but its output is much faster to read from due to the…
2019
- 5ML
Aug 2026 · github.com
- 6AM
2017 · github.com
- 7PO
Our company Vertex.AI has been working on this for a while but this is the first public release. We're starting with using PlaidML to bring OpenCL support to Keras and more frameworks, platforms, etc are coming. Yes, this means you can use use your AMD GPU for deep learning dev. Sorry, no Mac or Windows support yet although the brave can try building from source (it should work). http://vertex.ai/blog/announcing-plaidml https://github.com/plaidml/plaidml
2017
- 8PM
2018 · github.com
- 9LL
After the incredible response to our launch of the first online CUDA playground, we have just shipped something we think all you GPU programming and ML enthusiasts will love. Introducing LeetGPU Challenges--the place to compete on writing the fastest CUDA kernels. We have problems like matrix multiplication, agent simulation, multi-head self-attention, with more dropping every couple of days! We have a lot of really cool things coming up, including support for PyTorch, TensorFlow, JAX, TinyGrad; Multi-GPU programs; H100, V100, A100 GPU options Give it a try and let us know what you think!
2025 · leetgpu.com
- 10IW
Input a SMILES string (or pick one molecule from the examples) and it returns up to 100k molecules closest in 3-D shape or electrostatic similarity – from 10+ billion scale databases — typically in under 5-10 s. *Why it might interest HN* * Entire index lives on disk — no GPU at query-time, less than ~10 GB RAM total. * Built from scratch (no FAISS index / Milvus / Pinecone). * Index-build cost: one Nvidia T4 (~ 300USD) for one 5.5B database. * Open to anyone, predict ADMET, export results as CSV/SDF. Full write-up & benchmarks (DUD-E, LIT-PCBA, SVS) in the pre-print:…
2025 · cheese-new.deepmedchem.com
- 11WF
It's annoying having so many ML and GS training tools rely exclusively on cuda/nvidia . So for our open-source command-line gaussian splat converter/compressor, we decided to try WebGPU instead. It's working well so far and a single codebase runs on Linux, MacOS and Windows without too much fuss. This is mostly thanks to Google's dawn project <3. Eventually some of this could also run directly in the browser.
Sep 2025 · github.com
- 12AC
I've been building a C++ tensor library that tries to bring the ergonomics of NumPy/PyTorch to native code. The main goals were: - Familiar API (operator overloading, method chaining, same function names) - Actual GPU acceleration via Metal (not just matmul, but all ops) - Cross-platform with SIMD on x86/ARM/RISC-V via xsimd - einops-style rearrange/reduce patterns It started because I wanted NumPy's API but needed to deploy on edge devices without Python. Ended up going deeper than expected (28k LOC+) into BLAS backends, memory views, and GPU kernels. Some things I'm…
Feb 2026
- 13LS
Earlier this year, I took a month to reexamine my coding habits and rethink some past design choices. I hope to rewrite and improve my FOSS libraries this year, and I needed answers to a few questions first. Perhaps some of these questions will resonate with others in the community, too. - Are coroutines viable for high-performance work? - Should I use SIMD intrinsics for clarity or drop to assembly for easier library distribution? - Has hardware caught up with vectorized scatter/gather in AVX-512 & SVE? - How do secure enclaves & pointer tagging differ on Intel, Arm, & AMD? - What's…
2025 · github.com
- 14ES
Hi HN, I built EdgeVec, a vector database that runs entirely in the browser. It implements HNSW (Hierarchical Navigable Small World) graphs for approximate nearest neighbor search. Performance: - Sub-millisecond search at 100k vectors (768 dimensions, k=10) - 148 KB gzipped bundle - 3.6x memory reduction with scalar quantization Use cases: browser extensions with semantic search, local-first apps, privacy-preserving RAG. Technical: Written in Rust, compiled to WASM. Uses AVX2 SIMD on native, simd128 on WASM. IndexedDB for browser persistence. npm:…
Dec 2025 · github.com
- 15IE
Quick note on how it works and how I've done my batch embedding engine IgniteMS. The whole thing runs as one process using Rust, reading input, tokenizing, packing batches, keeping the queue full. TensorRT handles inference. Python is only as a wrapper. I built it this way because when you use more than couple of GPUs, the GPUs stop being the problem. CPU cannot feed them fast enough. One A100 can go through batches faster than Python can tokenize and feed, so the GPU just sits there idle waiting for work. Most of my time went into optimizing this. At 8 GPUs that was basically the entire…
Jun 2026 · github.com
- 16TP
I have had a few conversations in the past year with non-technical folks (traditional finance types, consultants) who asked for a simple explainer on how GPTs work. These people generally have the horsepower to grasp new concepts quickly but don't necessarily have the math background to dig in super deeply and want a stronger framework than "they predict words". I had generally been pointing them to Karpathy, Illustrated Transformer, and other youtube content but I figured something more hands on without the math requirement might be better at explaining questions like "why matrix…
Jun 2026 · understandgpt.xyz
- 17IT
2014 · blog.shriphani.com
- 18IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 195L
We've built InferX, a specialized runtime environment that fundamentally changes how LLMs are served. The core problem we solve is the latency bottleneck in AI inference, especially with large models. Current systems waste resources or suffer from painfully slow cold starts. InferX's AI-native architecture, with its "snapshot" technology, enables: * *Sub-2s cold starts:* Spin up models instantly. * *High density:* Serve more LLMs on the same GPUs. * *Optimal efficiency:* Maximize GPU utilization. This isn't just another API; it's a new execution layer designed from the ground up for the…
2025 · github.com
- 20RG
I wanted to know how fast a 26B mixture-of-experts model could run on a desktop CPU with no GPU. Got ~40 tok/s single-stream (lossless) and ~124 batched. The surprising part was the byte budget: for this model you compress the output head (32% of per-token bytes), not the experts (16%). The writeup has the bandwidth roofline and the dead-ends; the repo has the reproducible recipe. Happy to answer questions. Repo: https://github.com/arun-prasath2005/gemma4-cpu-moe
Jun 2026 · apeg.dev
- 21R0
I built a tiny physics solver LLM that performs surprisingly well on easy-to-medium difficulty physics problems. Most LLMs today still struggle with physics QA (as PhyBench recently highlighted), so I wanted to see how far I could push a small model with careful data and minimal compute. Model: Qwen3-1.7B Supervised Finetuning: ~1500 curated examples spanning kinematics, EM, acoustics, and more RL Fine-tuning: GRPO, 1-shot RLVR style (single example, 70 steps) Total cost: ~$5 on H100 It started with a cold-start SFT (~3 epochs, loss to 0.3), then I ran RL with accuracy reward that climbed…
2025 · huggingface.co
- 22
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
23d ago · github.com
- 23SL
I'd like to share a little toy project of mine, a really simple image codec that can do lossy to full lossless image compression with complete scalability at a byte level granularity - you can compress an image just once, even fully losslessly if needed, and then get any lossy version possible by simply stopping decompression at any offset in the compressed data. This "encode onde, serve many" approach is especially interesting for providing downscaled low quality image previews (LQIP) in as tight a storage budget as possible, and then allowing seamless, transparent refinement as deemed…
2024 · github.com
- 24
SVG optimizer - compress Figma exports 30%+ smaller in 1s
Jul 2026 · svgcompressor.pages.dev
Ranked by how close each launch is in meaning, then by votes. Refine with a description →