Llama.go – port of llama.cpp to pure Go
It's April 12 - The First Man in Space* Day - and I'm releasing #ML framework I've been building for the last month :) It's written in #Go and allows #LLaMA #GPT inference having just regular PC - so no monster GPU cluster is needed to start experiment with: https://github.com/gotzmann/llama.go The V1 is using FP32 math only, but will work with AVX2 data types and INT8 quantisation soon. * The first man in space was Yuri Gagarin from USSR
In plain words
Llama.go is a Go implementation of llama.cpp that enables LLaMA and GPT model inference on standard computers without requiring specialized GPU hardware. Written in pure Go, the framework allows developers and researchers to experiment with large language models on regular PCs. Version 1 uses FP32 math, with planned support for AVX2 data types and INT8 quantization coming soon.
written from the facts on this page · September 2026
Does the same job
all alternatives →
- LCLlama.cpp Tutorial 2026: Run GGUF Models Locally on CPU and GPUApr 2026 · ▲13
Complete llama.cpp tutorial for 2026. Install, compile with CUDA/Metal, run GGUF models, tune all inference flags, use the API server, speculative decoding, and benchmark your hardware. https://vucense.com/dev-corner/llama-cpp-tutorial-run-gguf-m...
- RARun any Llama model finetune and more, instantly2024 · featherless.ai · ▲7
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
- ICI co-wrote a book on training Deep Learning models in Go2019 · ▲45
A friend and I wrote a book on how to build and train Deep Learning models in Go. We wanted it to be a useful reference for deep learning basics for Go programmers. Deep Learning is slowly seeping into everything we use every day and we thought it would be great if more people could do it in Go. The book is available here and on Amazon as well. https://www.packtpub.com/big-data-and-business-intelligence/hands-deep-learning-go We would appreciate any feedback and we're always looking to improve.
- LKLLMKube – Kubernetes for Local LLMs with GPU AccelerationNov 2025 · github.com · ▲5
Hi HN! I built LLMKube, a Kubernetes operator for deploying GPU-accelerated LLMs in production. One command gets you from zero to inference with full observability. Why this exists: Regulated industries (healthcare, defense, finance) need air-gapped LLM deployments, but existing tools are either single-node only (Ollama) or lack GPU optimization and SLO enforcement. LLMKube bridges the gap. What's working: - 17x speedup with NVIDIA GPUs (64 tok/s on Llama 3.2 3B vs 4.6 tok/s CPU) - One command: llmkube deploy llama-3b --gpu (auto CUDA setup, scheduling, layer offloading) -…
- LLLow-latency local LLM runner via OpenJDK Panama FFM (Java 22)Jul 2026 · github.com · ▲38
I wanted to run AI from inside the JVM. I started out with the standard REST sidecar, ripped that out to use Project Panama (Foreign Function & Memory API) in the new JDK versions to interface directly with llama.cpp. I still wasn't happy with how that functioned, so I built libargus.cc to get a clean ABI to expose a structured API up in the JVM landscape. It still uses Project Panama to interface directly with llama.cpp, whisper.cpp, and ggml compute graphs. I have zero-allocation on the hot paths, memory segments for prompts and tokens are allocated once inside confined Arenas. Raw…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, April 2023
the whole month →
- G4
Hi HN, Today we’re launching GPT-4 answers on Phind.com, a developer-focused search engine that uses generative AI to browse the web and answer technical questions, complete with code examples and detailed explanations. Unlike vanilla GPT-4, Phind feeds in relevant websites and technical documentation, reducing the model’s hallucination and keeping it up-to-date. To use it, simply enable the “Expert” toggle before doing a search. GPT-4 is making a night-and-day difference in terms of answer quality. For a question like “How can I RLHF a LLaMa model”, Phind in Expert mode delivers a…
AI · 2023 · phind.com
