Alternatives
Products that do what Self-Hosted RAG with Llama 3 does
100% offline RAG for engineers who can't use cloud AI
- 1
- 2
- 3IB
Hey HN! Over the past few weeks, I’ve been working on DataBridge, an open-source solution for data ingestion and querying across text, PDFs, images, and videos. In our latest update, we’ve added a fully local deployment option: - No internet required – Runs entirely offline. - Customizable Models – Supports any LLM and embedding model via Ollama (with options for any other private providers) - Extensibility – You can plug in your own models or tools easily. This local-first approach ensures better privacy, security, and flexibility, especially for teams dealing with sensitive data. You can…
2025 · github.com
- 4AO
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com
- 5
- 6
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 7
- 8
- 9
- 10
- 11
- 12JS
Hey HN, I’m Julia, my team and I are building Rag-in-a-Box (https://www.joinable.ai/rag-in-a-box), hosted RAG service that let’s builders of any skill level launch their own RAG app loaded with their own data in minutes. [ What can you do ] 1. Load your documents (PDFs, CSV, PPTs, Word Docs, etc) and make them searchable instantly. All your data stays private and encrypted. 2. Choose latest open source LLM (Llama 4, Deepseek, GPT-oss, etc) to interact with your docs 3. Access your hosted RAG via API - build your own custom front end or integrate with your existing product…
2025 · joinable.ai
- 13
- 14
- 15SW
Hey HN, We’re Basia, Fokke, and Geno from Liquidmetal AI, and we built something we wish we had a long time ago: SmartBuckets. We’ve spent a lot of time building RAG and AI systems, and honestly, the infrastructure side has always been a pain. Every project turned into a mess of vector databases, graph databases, and endless custom pipelines before you could even get to the AI part. SmartBuckets is our take on fixing that. It works like an object store, but under the hood it handles the messy stuff — vector search, graph relationships, metadata indexing — the kind of infrastructure you'd…
2025
- 16
- 17IB
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
- 18CS
If you have developer documentation and want to boost your community with AI this is for you! Just pull in the base url of the site add some customization and get a sharable link for your chat, link it anywhere you want. I saw this trend in some places like gcp with Gemini, or Langchain or Supabase ask ai, but they're all custom-implemented solutions, not everyone wants to advocate developer resources to create the rag, deploy it and maintain it, you just want devs to build with your stuff, the more they can do the better, the quicker the better, and if they get a smooth experience while…
2024 · explainit.mzslabs.com
- 19RW
RAG Web UI is designed to be the most straightforward way to build your own knowledge-based Q&A system. While other RAG (Retrieval-Augmented Generation) projects might be complex, we focus on making it super easy to understand and use. Why It's The Most Beginner-Friendly: Simple Document Management - Just upload your documents (PDF, DOCX, Markdown, Text) - System handles all the complex processing automatically - No need to worry about document chunking or vectorization - Documents update automatically in the background Easy-to-Use Chat Interface - Ask questions in plain language - Get…
2025 · github.com
- 20BA
2023 · github.com
- 21WR
How It Works - Offline Indexing: Docs are processed and embedded using the GTE-small model at build time. Browser-Based Magic: - SQLite database (stored in the browser) for vector search. - Local embedding model for query processing. - Local LLaMA model for response generation using WebLLM. - Everything Happens Locally: No data leaves the user’s device. Key Benefits - No API Costs: Everything runs in the browser—zero backend expenses. - Unlimited Chats: No rate limits or usage restrictions. - Privacy-First: Your data stays on your device, always. You can find the code here:…
2024 · docs.akiradocs.ai
- 22OO
Hi HN, I built OpenFable, an open-source retrieval engine that implements the FABLE algorithm (https://arxiv.org/abs/2601.18116) for RAG pipelines. I'm using it in another project and thought that others might benefit. Most RAG systems chunk documents into flat segments and retrieve by vector similarity. This works for simple lookups but breaks when answers span multiple sections, when relevant content is buried in a subsection, or when you need to control how many tokens you're sending to an LLM. OpenFable takes a different approach: when you ingest a document, it uses…
Apr 2026 · github.com
- 23AL
Hi HN! I wanted to share my freshly finished open-source project. It is similar to ChatGPT Code Interpreter, but the interpreter runs locally and it can use open-source models like Llama 2. It allows you to work with sensitive data without uploading it to the cloud. Either you use a local LLM (like Llama 2), or an API (like GPT-4). For the latter case, there is an approval mechanism in the UI, which separates your local data from the remote services. I would be very interested in your valuable feedback!
2023 · github.com
- 24AO
Hi HN, We built one of the largest RAG set-ups that exist toady with Usul.ai (6B tokens). We started by using langchain and llamaindex, they were able to get us to a prototype in a couple of days, but took 3 months of taking pieces apart and optimizing them to make it perform well at such large scale. We put all of these learning into an MIT licensed open-source project — Agentset. Our goal to let people get production quality RAG w/o having to understand or optimize the underlying pieces. It supports 22 file formats, agentic search, deep research, citations, and a UI out of the box.…
Oct 2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →