AI · alternatives · 2026
24 alternatives to Self-Hosted RAG with Llama 3
100% offline RAG for engineers who can't use cloud AI
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1OS
Complete AI Platform: RAG Systems & Intelligent Agents for Local AI
2025 · rlama.dev · its alternatives →
- 2
Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information…
2023 · github.com · its alternatives →
- 3
- 4
RAGaaS▲75The API for building production-ready AI apps with your data
2025 · sitegpt.ai · its alternatives →
- 5
Llamao▲496Private & offline alternative to ChatGPT on your device
2025 · llamao.app · its alternatives →
- 6
- 7
- 8
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support! - getumbrel/llama-gpt
2023 · github.com · its alternatives →
- 9
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
2024 · embedding.io · its alternatives →
- 10
LiquidIndex 2.0▲134I'm building LiquidIndex, an RAG API service that makes it easy to add personalized context to AI apps. Instead of building a whole system to upload, process, and search through a user’s files or notes, you just call a few APIs. It handles the hard parts - like connecting to Notion or Google Drive, breaking the data into usable pieces, and making it searchable. Then, when you want to answer a question using that data, you just query it, and it gives you the most relevant results - optionally with an LLM-generated answer. No pipelines, no infrastructure, no headaches. Here are the core APIs:…
2025 · liquidindex.dev · its alternatives →
- 11
Hi HN A few folks and I have been working on this project for a couple weeks now. After previously working on the Docker project for a number of years (both on the container runtime and image registry side), the recent rise in open source language models made us think something similar needed to exist for large language models too. While not exactly the same as running linux containers, running LLMs shares quite a few of the same challenges. There are "base layers" (e.g. models like Llama 2), specific configuration to run correctly (parameters, temperature, context window sizes etc). There's…
2023 · github.com · its alternatives →
- 12
Hey everyone, We have been building SecureAI Tools -- an open-source application layer for ChatGPT and ChatPDF-like AI tools. It works with locally running LLMs as well as with OpenAI-compatible APIs. For local LLMs, it supports Ollama which supports all the gguf/ggml models. Currently, it has two features: Chat-with-LLM, and Chat-with-PDFs. It is optimized for self-hosting use cases and comes with basic user management features. Here are some quick demos: * Chat with documents using OpenAI's GPT3.5 model: https://www.youtube.com/watch?v=Br2D3G9O47s * Chat with documents…
2023 · github.com · its alternatives →
- 13
Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/. Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year. Problems Ragas can solve - How do you choose the best components for your RAG, such as the retriever, reranker, and LLM? - How do you formulate a test dataset…
2024 · github.com · its alternatives →
- 14
Hey HN! Over the past few weeks, I’ve been working on DataBridge, an open-source solution for data ingestion and querying across text, PDFs, images, and videos. In our latest update, we’ve added a fully local deployment option: - No internet required – Runs entirely offline. - Customizable Models – Supports any LLM and embedding model via Ollama (with options for any other private providers) - Extensibility – You can plug in your own models or tools easily. This local-first approach ensures better privacy, security, and flexibility, especially for teams dealing with sensitive data. You can…
2025 · github.com · its alternatives →
- 15
TL;DR: Vector-based RAG performs poorly for many real-world applications like codebase chats, and you should consider 'language maps'. Part of our mission at Mutable.ai is to make it much easier for developers to build and understand software. One of the natural ways to do this is to create a codebase chat, that answer questions about your repo and help you build features. It might seem simple to plug in your codebase into a state-of-the-art LLM, but LLMs have two limitations that make human-level assistance with code difficult: 1. They currently have context windows that are too small to…
2024 · twitter.com · its alternatives →
- 16
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com · its alternatives →
- 17
- 18
Dabarqus▲92Add PDF chat to your LLM app in less than 9 lines of code
2024 · dabarqus.com · its alternatives →
- 19
Hey HN, exciting news! Our RAG framework, Cognita (https://github.com/truefoundry/cognita), born from collaborations with diverse enterprises, is now open-source. Currently, it offers seamless integrations with Qdrant and SingleStore. In recent weeks, numerous engineers have explored Cognita, providing invaluable insights and feedback. We deeply appreciate your input and encourage ongoing dialogue (share your thoughts in the comments – let's keep this ‘open source’). While RAG is undoubtedly powerful, the process of building a functional application with it can feel…
2024 · github.com · its alternatives →
- 20
- 21
Axis One▲78I’ve been building a small hardware box that runs local LLMs like Mistral, Qwen and Llama, and exposes an OpenAI compatible API on your local network. There’s no cloud, no login system and no telemetry. I built it because a lot of small firms want ChatGPTbstyle tools but can’t use cloud AI for privacy or compliance reasons, and most don’t want to deal with GPU servers, drivers, Docker or model configs. The aim is to make local AI feel as simple as plugging in a router. Right now the box boots into a very simple web UI where you choose a model and start using it. The API follows the OpenAI…
Nov 2025 · axis-one-psi.vercel.app · its alternatives →
- 22LW
Hey HN! We are happy to share with you our latest project: Llama Workspace (https://llamaworkspace.ai). It's an open-source ChatGPT Teams alternative made with the needs of businesses and organizations in mind. It has everything you can expect from ChatGPT, plus first class support for users and roles management, as well as advanced collaboration features. We've made it very easy to self-host anywhere you like, and we also provide a cloud version. So no excuse for trying it out! Why using Llama Workspace instead of ChatGPT Teams? - You have access to all the major Large Language…
2024 · llamaworkspace.ai · its alternatives →
- 23QP
Hey HN! We've just launched Quilt, a robust RAG (Retrieval-Augmented Generation) UI that revolutionizes how you interact with your documents. Key features: - Multi-user setup with private/public document collections - Advanced hybrid RAG pipeline combining full-text & vector search - Smart citations with in-browser PDF preview and highlights - Fully customizable settings and prompts through the UI Making an account is free, no need to even use a strong password: this is only to ensure your documents are separate from the rest. We're keen to hear your thoughts and feedback. What features…
2024 · quilt.fly.dev · its alternatives →
- 24
Ragas is an open-source library designed for evaluating and testing RAG (Retrieval-Augmented Generation) and other LLM applications. It offers a diverse set of metrics and methods, including synthetic test data generation, to help you assess your RAG applications. Ragas was initially developed to address our own needs for evaluating RAG chatbots last year. ### Problems Ragas Can Solve: - How can you select the best components for your RAG, such as the retriever, reranker, and LLM? - How can you create a test dataset without incurring significant expenses and time? We believe there's a need…
2024 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →