AI · alternatives · 2026
24 alternatives to L88 – A Local RAG System on 8GB VRAM (Need Architecture Feedback)
Hey everyone, I’ve been working on a project called L88 — a local RAG system that I initially focused on UI/UX for, so the retrieval and model architecture still need proper refinement. Repo: https://github.com/Hundred-Trillion/L88-Full I’m running this on 8GB VRAM and a strong CPU (128GB RAM). Embeddings and preprocessing run on CPU, and the main model runs on GPU. One limitation I ran into is that my evaluator and generator LLM ended up being the same model due to compute constraints, which defeats the purpose of evaluation. I’d really appreciate feedback on:…
L88 is a local retrieval-augmented generation system designed to run on modest hardware, using 8GB of GPU VRAM with CPU-based embeddings and preprocessing. It is for developers interested in building RAG systems on… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm
May 2026 · github.com · its alternatives →
- 2
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup
Aug 2026 · github.com · its alternatives →
- 3
The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
2024 · github.com · its alternatives →
- 4
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
- 5WM
Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…
2024 · glhf.chat · its alternatives →
- 6
Ollama v0.19▲410Massive local model speedup on Apple Silicon with MLX
Apr 2026 · ollama.com · its alternatives →
- 7
Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?" This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer gpus, it should work better on professional ones tho
Feb 2026 · github.com · its alternatives →
- 8
Contribute to Mega4alik/ollm development by creating an account on GitHub.
2025 · github.com · its alternatives →
- 9
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai · its alternatives →
- 10
- 11
I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
2023 · github.com · its alternatives →
- 12
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
- 13
Hi HN, I’m Joe. My friends Matthew, Jake and I are building Luminal (https://luminalai.com/), a GPU compiler for automatically generating fast GPU kernels for AI models. It uses search-based compilation to achieve high performance. We take high level model code, like you'd have in PyTorch, and generate very fast GPU code. We do that without using LLMs or AI - rather, we pose it as a search problem. Our compiler builds a search space, generates millions of possible kernels, and then searches through it to minimize runtime. You can try out a demo in `demos/matmul` on mac to…
2025 · github.com · its alternatives →
- 14QA
2017 · anchovieshat.github.io · its alternatives →
- 15
I built a new 8-bit CPU in VHDL from scratch (starting from the ISA). I felt that most educational soft-cores hide too much behind abstraction, eg. if I can do a+b with a single assignment that calls an optimized arithmetic library, then why did I learn the ripple carry adder in the first place ? And why did I learn flip flops if I can do all my control logic with a simple PROCESS statement like I would with a programming language ? Of course abstraction is the main selling point of HDLs, but would it work if I tried to keep strictly structural and rely on ieee.std_logic_1164 only ? Well, it…
Jan 2026 · github.com · its alternatives →
- 16IM
2023 · vram.asmirnov.xyz · its alternatives →
- 17
GitLab Community Edition
2014 · gitlab.brokenpipe.de · its alternatives →
- 18CI
One of the most frequent questions one faces while running LLMs locally is: I have xx RAM and yy GPU, Can I run zz LLM model ? I have vibe coded a simple application to help you with just that. Update: A lot of great feedback for me to improve the app. Thank you all.
2025 · can-i-run-this-llm-blue.vercel.app · its alternatives →
- 19
How to run Llama3.1 405B on a 8GB VRAM
2024 · youtube.com · its alternatives →
- 20
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com · its alternatives →
- 21
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com · its alternatives →
- 22
High performance storage engine for efficient LLM inference and GPU Training.
Sep 2026 · theopenlake.com · its alternatives →
- 23IB
Hey all! I wanted to share this project I've been working on that can maybe help you or your developer friends out. I built a RAG system for our product a while back and didn't realize how easy they were to get started. So I put together my learnings into this online course. It's not quite ready but if you sign up and mention HackerNews, I can get you early access. I'm looking to get feedback on the following: (1) materials — is it engaging & did you learn something? (2) UI/UX of the platform — did you have any issues that prevented you from starting or finishing the tutorial? (3)…
2024 · takehomes.com · its alternatives →
- 24UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024 · its alternatives →
Also compare
- Find the best local LLM for your hardware, ranked by benchmarks alternatives
- Fine-tune an 8B model on a 4 GB laptop GPU alternatives
- I've built a locally running Perplexity clone alternatives
- 80% faster, 50% less memory, 0% loss of accuracy Llama finetuning alternatives
- We made glhf.chat – run almost any open-source LLM, including 405B alternatives
- Ollama v0.19 alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →