AI · alternatives · 2026

24 alternatives to Llama.go – port of llama.cpp to pure Go

It's April 12 - The First Man in Space* Day - and I'm releasing #ML framework I've been building for the last month :) It's written in #Go and allows #LLaMA #GPT inference having just regular PC - so no monster GPU cluster is needed to start experiment with: https://github.com/gotzmann/llama.go The V1 is using FP32 math only, but will work with AVX2 data types and INT8 quantisation soon. * The first man in space was Yuri Gagarin from USSR

Llama.go is a Go implementation of llama.cpp that enables LLaMA and GPT model inference on standard computers without requiring specialized GPU hardware. Written in pure Go, the framework allows developers and… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Llama.go – port of llama.cpp to pure Go launched in 2023; newer entries below may have overtaken it.

1
Llama▲312
3.1-405B: an open source model to rival GPT-4o / Claude-3.5
2024 · developer.meta.com · its alternatives →
2
Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model▲343
High-speed download of LLaMA, Facebook's 65B parameter GPT model - shawwn/llama-dl
2023 · github.com · its alternatives →
3
Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU▲395
Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?" This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer gpus, it should work better on professional ones tho
Feb 2026 · github.com · its alternatives →
4
Llama 2▲263
The next generation of Meta's open source LLM
2023 · developer.meta.com · its alternatives →
5
Llamao▲496
Private & offline alternative to ChatGPT on your device
2025 · llamao.app · its alternatives →
6
Finetune LLaMA-7B on commodity GPUs using your own text▲449
I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
2023 · github.com · its alternatives →
7
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning▲385
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
8
Tune LLaMa3.1 on Google Cloud TPUs▲189
Hey HN, we wanted to share our repo where we fine-tuned Llama 3.1 on Google TPUs. We’re building AI infra to fine-tune and serve LLMs on non-NVIDIA GPUs (TPUs, Trainium, AMD GPUs). The problem: Right now, 90% of LLM workloads run on NVIDIA GPUs, but there are equally powerful and more cost-effective alternatives out there. For example, training and serving Llama 3.1 on Google TPUs is about 30% cheaper than NVIDIA GPUs. But developer tooling for non-NVIDIA chipsets is lacking. We felt this pain ourselves. We initially tried using PyTorch XLA to train Llama 3.1 on TPUs, but it was rough: xla…
2024 · github.com · its alternatives →
9
Speeding up LLM inference 2x times (possibly)▲419
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
10
Llama 3.3 70B▲347
Llama 405B-level performance, at a fraction of the cost
2024 · developer.meta.com · its alternatives →
11
LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2▲225
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support! - getumbrel/llama-gpt
2023 · github.com · its alternatives →
12LC
Llama.cpp Tutorial 2026: Run GGUF Models Locally on CPU and GPU▲13
Complete llama.cpp tutorial for 2026. Install, compile with CUDA/Metal, run GGUF models, tune all inference flags, use the API server, speculative decoding, and benchmark your hardware. https://vucense.com/dev-corner/llama-cpp-tutorial-run-gguf-m...
Apr 2026 · its alternatives →
13
Ollama▲235
The easiest way to run large language models locally
2023 · github.com · its alternatives →
14
NobodyWho▲106
Hey, Hello, we are the team behind NobodyWho; an open-source plugin that embeds large language models functionality inside Unity and Godot games - completely locally and offline. We try to make it as easy and user friendly as possible to use our plugin, so feel free to create an issue on our GitHub if you have any inconveniences - and we will try to improve it! So why might this interest you: - Written in Rust with llama.cpp bindings and optional GPU back-ends (Vulkan / Metal) for fast inference. - Drop-in nodes / components: add a model asset and a chat or embedding node and get a…
Aug 2026 · github.com · its alternatives →
15
A fully open-source (Apache 2.0)implementation of llama▲158
We believe that AI should be fully open source and part of the collective knowledge. The original LLaMA code is GPL licensed which means any project using it must also be released under GPL. This "taints" any other code and prevents meaningful academic and commercial use. Lit-LLaMA solves that for good.
2023 · github.com · its alternatives →
16
LLaMA▲118
A foundational, 65-billion-parameter large language model
2023 · research.facebook.com · its alternatives →
17
Llama-8B Teaches Itself Baby Steps to Deep Research Using RL▲39
I've been tinkering with getting Llama-8B to bootstrap its own research skills through self-play. The model generates questions about documents, searches for answers, and then learns from its own successes/failures through RL (hacked up Unsloth's GRPO code). Started with just 23% accuracy on Apollo 13 mission report questions and hit 53% after less than an hour of training. Everything runs locally using open-source models. It's cool to see the model go from completely botching search queries to iteratively researching to get the right answer.
2025 · github.com · its alternatives →
18
TinyGo – LLVM-based Go compiler for microcontrollers▲177
Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. - tinygo-org/tinygo
2018 · github.com · its alternatives →
19
LLaVaVision: An AI "Be My Eyes"-like web app with a llama.cpp backend▲154
A simple mobile web app inspired by Fuzzy-Search/realtime-bakllava that uses llama.cpp server backend with multimodal mode to describe and narrate what the phone camera sees. I built this thing in a few hours using a single ChatGPT thread to generate most things for me and iterate on this project. Here's the workflow: https://chat.openai.com/share/ea84ec69-5617-45e8-8772-ac2dcf...
2023 · github.com · its alternatives →
20
Open-source load balancer for llama.cpp▲151
Stateful load balancer customized for llama.cpp (with a reverse proxy).
2024 · github.com · its alternatives →
21
Local LLM Notepad – run a GPT-style model from a USB stick▲40
What it is A single 45 MB Windows .exe that embeds llama.cpp and a minimal Tk UI. Copy it (plus any .gguf model) to a flash drive, double-click on any Windows PC, and you’re chatting with an LLM—no admin rights, Cloud, or network. Why I built it Existing “local LLM” GUIs assume you can pip install, pass long CLI flags, or download GBs of extras. I wanted something my less-technical colleagues could run during a client visit by literally plugging in a USB drive. How it works PyInstaller one-file build → bundles Python runtime, llama_cpp_python, and the UI into a single PE. On first launch, it…
2025 · github.com · its alternatives →
22AT
A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)▲55
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
23
Llama Running on a Microcontroller▲49
A "large" language model running on a microcontroller - maxbbraun/llama4micro
2023 · github.com · its alternatives →
24
TurboPilot: Copilot clone runs code completion LLM on your CPU▲37
Hi HN, I spent my easter weekend stuck in the house with COVID and I decided to play with llama.cpp [1] and fauxpilot [2] to see if I could get LLM code assist working on pure CPU. As a proof of concept I'd say I've proven that it's possible. However there's still a lot to do. The auto complete is quite slow at the moment. PRs welcome. [1] https://github.com/ggerganov/llama.cpp [2] https://github.com/fauxpilot/fauxpilot
2023 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →