Dev tools · alternatives · 2026

24 alternatives to Unfat, a library to easily train and distill LoRAs for LLMs

Hey HN! After using a combination of Unsloth and Axolotl a lot, and finding it generally painful to figure out the right performance tuning for things like batch sizing and multi-GPU sharding, I wrote a small Python lib that sets up known-good LoRA training configurations for Llama 3.1 8B and 70B Instruct, and includes helpers for distilling from larger models or training on serverless finetuning platforms, and includes a walkthrough for distilling DeepSeek-R1 into a Llama 3.1 8B LoRA... But you can use it for pretty much any finetuning task, not just distilling large models!

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
Finetune LLaMA-7B on commodity GPUs using your own text▲449
I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
2023 · github.com · its alternatives →
2
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning▲385
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
3
TensorZero▲307
Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
4
Unsloth▲241
Finetune LLMs 2x faster, 80% less memory
2025 · unsloth.ai · its alternatives →
5
SelfHostLLM▲134
A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
6IB
I built a website where you can easily fine-tune Llama 3.1 models▲8
Hey HN, I built a website where you can train Llama 3.1 8b & 70b (4bit) on your data. I use unsloth in the backend and the training is done on H100s which I rent programmatically from Runpod. I'd love some feedback. If you would be interested in using it feel free to book a chat with me: cal.com/hamada/tunellama-intro Happy to give you free credits :) P.S. I'm also looking for a co-founder as I have big plans for this.
2024 · tunellama.com · its alternatives →
7
finetune LLMs via the Finetuning Hub▲80
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for…
2023 · github.com · its alternatives →
8
AnyLLM▲20
10+ LLMs at 10x speed. Think we're kidding. Test it now!
2025 · its alternatives →
9
Zipslicer, a library for loading LLM checkpoints on consumer hardware▲38
This is a low-level opensource library I developed for my own use and decided to share, as it makes it possible to process large checkpoints of neural networks without renting high-RAM instances, on a regular PC. It replaces torch.load() with a custom function that produces a dictionary that materializes tensors on the fly. Compared to other solutions it doesn't require sharding or re-encoding checkpoints and uses them completely as-is. It is a foundation to make it possible to run inference and compress language models and other large models one layer at a time - in principle, even one…
2023 · github.com · its alternatives →
10
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
11
DistilKitPlus, a distillation framework between any LLMs▲10
Over the past few months, I have built a distillation toolkit that supports cross-tokenizer distillation (e.g., distilling from LLaMA to Qwen vocab, or others). This approach has worked well on reasoning datasets like AIME, and we’ve validated on models like Phi and Qwen. We’ve also integrated Modal for quick deployment (with $30/month credits to try it out). Would love any feedback! GitHub: https://github.com/agokrani/distillKitPlus Docs: https://distillkitplus.mintlify.app/
2025 · github.com · its alternatives →
12IB
I built a local Elixir/Python pipeline to curate 14,000 RAW photos▲5
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com · its alternatives →
13
Gradient – a web API for fine-tuning and deploying Llama2▲26
We’ve just launched Gradient — an API that helps you build private LLMs that you own. We simplify inference and fine-tuning on open-source LLMs such as llama2, and you only pay by the token. Our API platform makes it possible for you to create private models with a single API call. Run inference on your fine tuned model instantly with no cold boot (and no need to pay for compute costs). The product is truly on demand - when you run fine tuning and inference on our platform, there's nearly 0 startup latency for these API calls. And you're not paying for the compute, you just pay for the…
2023 · gradient.ai · its alternatives →
14
Run any Llama model finetune and more, instantly▲7
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai · its alternatives →
15
We built the Keras of LLMs▲5
Here is a production-first Keras-inspired LM framework, built with the advice of François Chollet (ex-Google, creator of Keras and ARC-AGI), our technical advisor. This system have already been deployed in production with our clients (which is why we have already every LLMOps practice implemented). It is also compatible with Jupyter and Marimo to integrate seamlessly in you Data Scientists workflows. You can try the code examples online on HF space and you can find more information in the documentation and FAQ. If you have any feedback for us don't hesitate to join our discord! More releases…
2025 · github.com · its alternatives →
16
KV-psi, using Linux PSI to to trim an LLM KV cache▲8
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com · its alternatives →
17
Plock: Use a local LLM from anywhere in your OS▲6
After seeing a cool demo of a hack on Twitter, I built a cross platform version of it that works well and uses streaming. From anywhere on Mac and Linux, trigger Ollama and optionally feed it your clipboard. I built it yesterday and it's already very useful to me. I'm pretty excited about it and wanted to share!
2024 · github.com · its alternatives →
18
An educational Local Qwen3 LLM Inference project written in Rust▲15
An educational Rust project for exporting and running inference on Qwen3 LLM family - reinterpretcat/qwen3-rs
2025 · github.com · its alternatives →
19HF
Higgsfield – Finetune LLMs for Free▲5
We have a massive GPU cluster and developed our own infrastructure to manage the cluster and train massive models. There's how it works: 1. You upload the dataset with preconfigured format into HuggingFaсe [1]. 2. Choose your LLM (e.g. LLaMa 70B, Mistral 7B) 3. Place your submission into the queue 4. Wait for it to get trained. 5. Then you get your trained model there on HuggingFace. Essentially, why would we want to do it? 1. We already have an experience with training big LLMs. 2. We could achieve near-perfect infrastructure performance for training. 3. Sometimes GPUs have just nothing to…
2023 · higgsfield.xyz · its alternatives →
20
LLGTRT: TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output▲6
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com · its alternatives →
21
Toolkit for LLM Fine-Tuning, Ablating and Testing▲14
Hello all! Very happy to share this toolkit that allows you to fine-tune your choice of open-source LLMs on your data! The toolkit also allows you to run ablation studies across LLMs, prompt designs, training configurations, and can ingest different data files -- all through just ONE YAML file! After fine-tuning, you can also run a bunch of tests to ensure that the fine-tuned LLM behaves as expected, enabling faster time-to-production! Why this toolkit? Why now? While closed-source LLMs have become popular for chat-based applications, enterprises are considering a shift to self-hosted SLMs…
2024 · github.com · its alternatives →
22
Open-source fine-tuning in a Colab notebook▲5
Posted before, but wanted to share if you want an open source alternative to OpenAI fine-tuning, give Unsloth a try! Phi 3.5 was just released, and is distilled from GPT4. Unsloth makes finetuning 2x faster, uses 70% less VRAM + has no accuracy degradations. We rewrite all backprop steps and reduce FLOPs and write everything in Triton (JIT low level CUDA). If you want to own the weights after fine-tuning, give Unsloth a spin! I have free Colabs and Kaggle notebooks as well at https://github.com/unslothai/unsloth
2024 · colab.research.google.com · its alternatives →
23
I trained a 65B LLM on my texts to talk to myself (details inside)▲13
I trained the 65b model on my texts so I can talk to myself. It's pretty useless as an assistant, and will only do stuff you convince it to, but I guess it's technically uncensored? I'll leave it up for a bit if you want to chat with it. I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion: LLAMA 1 65b Rank 128 5 epochs Batch size 1, 256 cutoff Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora Loss around 1.5 seems to give the most coherent results Trained on raw text dumps that is then parsed by a crappy…
2023 · airic.serveo.net · its alternatives →
24
BonzAI – self-sovereign, local LLM inference in the browser▲5
Hi everyone, I started building BonzAI two years ago, as an attempt to fully replace my online AI stack with an offline, sovereign alternative. Back in the days, I had the intuition that open weights model would stand the comparison with private AI labs, and that the commoditization of inference was the future. When I was finally able to switch from ChatGPT/Claude to local LLMs, midjourney to Flux/SDXL/Z-image, Eleven Labs/Suno to Qwen-TTS/Ace-Step, and Sora to LTX, I implemented: - The ability to create datasets and fine-tune LoRAs; - An experimental 3D objects…
May 2026 · bonzai.sh · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →