AI · alternatives · 2026
24 alternatives to Toolkit for LLM Fine-Tuning, Ablating and Testing
Hello all! Very happy to share this toolkit that allows you to fine-tune your choice of open-source LLMs on your data! The toolkit also allows you to run ablation studies across LLMs, prompt designs, training configurations, and can ingest different data files -- all through just ONE YAML file! After fine-tuning, you can also run a bunch of tests to ensure that the fine-tuned LLM behaves as expected, enabling faster time-to-production! Why this toolkit? Why now? While closed-source LLMs have become popular for chat-based applications, enterprises are considering a shift to self-hosted SLMs…
This toolkit enables users to fine-tune open-source language models on custom data through a single YAML configuration file. It supports ablation studies across different models, prompts, and training settings, and can… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier. Toolkit for LLM Fine-Tuning, Ablating and Testing launched in 2024; newer entries below may have overtaken it.
- 1
I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
2023 · github.com · its alternatives →
- 2
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup
Aug 2026 · github.com · its alternatives →
- 3
Hey HN! We’re building FinetuneDB (https://finetunedb.com/), an LLM fine-tuning platform. It enables teams to easily create and manage high-quality datasets, and streamlines the entire workflow from fine-tuning to serving and evaluating models with domain experts. You can check out our docs here: (https://docs.finetunedb.com/) FinetuneDB exists because creating and managing high-quality datasets is a real bottleneck when fine-tuning LLMs. The quality of your data directly impacts the performance of your fine-tuned models, and existing tools didn’t offer an easy…
2024 · finetunedb.com · its alternatives →
- 4
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 5
- 6
LLM Stats▲293I was overspending on GPT-4o. It was really hard to compare different models I could switch to, so I built this LLM comparison tool. It shows leaderboards, pricing, and performance data across 100+ LLMs (including all major providers and open-source models). Key features: - Live pricing comparisons - Benchmark Scores (MMLU, HumanEval, GPQA, etc.) - Context length vs cost analysis - Speed/throughput tests across providers - Quality vs price visualizations - Open source (all data verifiable) Try it out: https://llmstats.com I'd like to know your opinion :) Tech stack: Next.js,…
Oct 2025 · llm-stats.com · its alternatives →
- 7
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for…
2023 · github.com · its alternatives →
- 8
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://arxiv.org/abs/2311.14479. Feedback and potential applications are welcome.
2023 · github.com · its alternatives →
- 9
Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm
May 2026 · github.com · its alternatives →
- 10AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 11
LLM reinforcement fine-tuning platform to improve LLM output
2025 · rubrik.com · its alternatives →
- 12
We’ve built a LLM fine-tuning SDK that supports Llama 3.1 right now. We’d love your feedback, and are happy to give extra free credits if you try the product and jump on a 15 minute call and provide feedback :)
2024 · docs.luminolabs.ai · its alternatives →
- 13
- 14
LLM Gatewayfree tier▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
- 15
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
- 16
TuneTrain.ai▲79Do you know that Small Language Models (SLM) can outperform LLMs if trained (fine-tuned) on domain specific datasets, like your companies knowledge base? The issue: fine-tuning a SLM usually requires a dataset of 10k - 100k records, which is huge. I created a platform for applying "human in the loop" augmentation techniques on your small dataset, so that you can start with maybe 100 records and build-up quickly huge datasets and launch a training without prior knowledge. I implemented 2 techniques, based on LLM distillation, however: more to come. HN, what Do you think? Do you see value in…
2025 · tunetrain.ai · its alternatives →
- 17
Hi everyone, we’re a small team, supported by Mozilla, who are working on re-imagining a UI for training, tuning and testing local LLMs. Everything is open source. If you’ve been training your own LLMs or have always wanted to, we’d love for you to play with the tool and give feedback on what the future development experience for LLM engineering could look like.
2025 · github.com · its alternatives →
- 18
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM. It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization It's a pretty…
2023 · docs.helix.ml · its alternatives →
- 19
Hey HN, I made Browser-Use, an open-source tool that lets (all Langchain supported) LLMs execute tasks directly in the browser just with function calling. It allows you to build agents that interact with web elements using natural language prompts. We created a layer that simplifies website interaction for LLMs by extracting xPaths and interactive elements like buttons and input fields (and other fancy things). This enables you to design custom web automation and scraping functions without manual inspection through DevTools. Hasn't this been done a lot of times? Good question, as a general…
2024 · github.com · its alternatives →
- 20
OpenSheet▲86Hi folks. I've been doing some experiments on how LLMs could get more handy in the day to day of working with files (CSV, Parquet, etc). Earlier last year, I built https://datakit.page and evolved it over and over into an all in-browser experience with help of duckdb-wasm. Got loads of feedbacks and I think it turned into a good shape with being an adhoc local data studio, but I kept hearing two main things/issues: Why can't the AI also change cells in the file we give to it? Why can't we modify this grid ourselves? So besides the whole READ and text-to-SQL flows, what seemed…
Feb 2026 · opensheet.app · its alternatives →
- 21
Arkor▲142Fine-tune and Deploy Open-weight Models in TypeScript
Jul 2026 · arkor.ai · its alternatives →
- 22UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024 · its alternatives →
- 23
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com · its alternatives →
- 24
Hey HN, we're super excited to share something we've been working on: OpenLIT. After an engaging preview that some of you might recall, we are now proudly announcing our first stable release! *What's OpenLIT?* Simply put, OpenLIT is an open-source tool designed to make monitoring your Large Language Model (LLM) applications straightforward. It’s built on OpenTelemetry, aiming to reduce the complexities that come with observing the behavior and usage of your LLM stack. *Beyond Basic Text Generation:* OpenLIT isn’t restricted to just text and chatbot outputs. It now includes automatic…
2024 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →