AI · alternatives · 2026
24 alternatives to FlashMLA
Faster LLM Inference on Hopper GPUs
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
- 2
I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
2023 · github.com · its alternatives →
- 3
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
- 4
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
- 5
Ollama v0.19▲410Massive local model speedup on Apple Silicon with MLX
Apr 2026 · ollama.com · its alternatives →
- 6
DeepSeek-V4▲387The open-source era of 1M context intelligence
Apr 2026 · huggingface.co · its alternatives →
- 7
DeepSeek-R1-0528▲218New open-source LLM that rivals o3 in coding & reasoning
2025 · huggingface.co · its alternatives →
- 8
Long-context efficiency with DeepSeek Sparse Attention
2025 · huggingface.co · its alternatives →
- 9
Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference. - kossisoroyce/timber
Mar 2026 · github.com · its alternatives →
- 10
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 11
- 12
- 13
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup
Aug 2026 · github.com · its alternatives →
- 14
- 15
SelfHostLLM▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
- 16AT
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
Aug 2026 · mikeayles.com · its alternatives →
- 17
This is a low-level opensource library I developed for my own use and decided to share, as it makes it possible to process large checkpoints of neural networks without renting high-RAM instances, on a regular PC. It replaces torch.load() with a custom function that produces a dictionary that materializes tensors on the fly. Compared to other solutions it doesn't require sharding or re-encoding checkpoints and uses them completely as-is. It is a foundation to make it possible to run inference and compress language models and other large models one layer at a time - in principle, even one…
2023 · github.com · its alternatives →
- 18
Working on Mac, Linux, and Windows now. I include a simple GUI to find new models and get things built and set up. It is working quite well across a few models for me. The GitHub README and DESIGN.md files go into detail of the how/why and it's working remarkably well so far. https://github.com/notactuallytreyanastasio/shoehorn
Aug 2026 · notactuallytreyanastasio.github.io · its alternatives →
- 19AG
This is a vector index I built that supports insertion and k-nearest neighbors (k-NN) querying, optimized for GPUs. It operates entirely in CUDA and can process queries on half a billion vectors in under 200 milliseconds. The codebase is structured as a standalone library with an HTTP API for remote access. It’s intended for high-performance search tasks—think similarity search, AI model retrieval, or reinforcement learning replay buffers. The codebase is located at https://github.com/rodlaf/BinaryGPUIndex.
2025 · rlafuente.com · its alternatives →
- 20
High performance storage engine for efficient LLM inference and GPU Training.
Sep 2026 · theopenlake.com · its alternatives →
- 21PO
Our company Vertex.AI has been working on this for a while but this is the first public release. We're starting with using PlaidML to bring OpenCL support to Keras and more frameworks, platforms, etc are coming. Yes, this means you can use use your AMD GPU for deep learning dev. Sorry, no Mac or Windows support yet although the brave can try building from source (it should work). http://vertex.ai/blog/announcing-plaidml https://github.com/plaidml/plaidml
2017 · its alternatives →
- 22
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM. It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization It's a pretty…
2023 · docs.helix.ml · its alternatives →
- 23
Contribute to seiflotfy/pcsa development by creating an account on GitHub.
2018 · github.com · its alternatives →
- 24
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com · its alternatives →
Also compare
- Tiny-vLLM – high performance LLM inference engine in C++ and CUDA alternatives
- Finetune LLaMA-7B on commodity GPUs using your own text alternatives
- Speeding up LLM inference 2x times (possibly) alternatives
- 80% faster, 50% less memory, 0% loss of accuracy Llama finetuning alternatives
- Ollama v0.19 alternatives
- DeepSeek-V4 alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →