Airgapped Offline RAG – Run LLMs Locally with Llama, Mistral, & Gemini
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
Airgapped Offline RAG – Run LLMs Locally with Llama, Mistral, & Gemini launched on October 11, 2024 with 9 votes, #1,236 of 5,185 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.
What it does
Secure, locally-run Retrieval-Augmented Generation system for document-based question-answering, utilizing Llama 3, Mistral, and Gemini models with a user-friendly Streamlit interface. - vincentkoc/airgapped-offfline-rag
Does a similar job
all alternatives →
Demystifying Advanced RAG Pipelines2023 · github.com · ▲131I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
I built a free in-browser Llama 3 chatbot powered by WebGPU2024 · github.com · ▲547I spent the last few days building out a nicer ChatGPT-like interface to use Mistral 7B and Llama 3 fully within a browser (no deps and installs). I’ve used the WebLLM project by MLC AI for a while to interact with LLMs in the browser when handling sensitive data but I found their UI quite lacking for serious use so I built a much better interface around WebLLM. I’ve been using it as a therapist and coach. And it’s wonderful knowing that my personal information never leaves my local computer. Should work on Desktop with Chrome or Edge. Other browsers are adding WebGPU support as well - see…
Turn any website into a knowledge base for LLMs2024 · embedding.io · ▲305I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
RAGstack – private ChatGPT for enterprise VPCs, built with Llama 22023 · github.com · ▲84Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information…
Mockingbird is an LLM that outperforms GPT4 on RAG2024 · vectara.com · ▲9Hi HN! I lead product at Vectara and we've just released a new LLM in our platform that outperforms GPT4 and Gemini 1.5 Pro on RAG tasks. Vectara is a Retrieval Augmented Generation (RAG) platform primarily deployed as a SaaS service which includes a generous free tier so you can try it for free. The way we've been able to offer a "better but cheaper" is that we focus a lot of our attention on taking smaller models (which can be hosted in a cost efficient way) and fine tuning them to specific tasks: in this case RAG. This ends up with a model that is less capable of arbitrary tasks like…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
Launched alongside, October 2024
the whole month →
bolt.new▲1,230Build and scale high-performing websites & apps using your words. Join millions and start building today.
AI · 2024 · bolt.new
Softr for Notion▲923Turn Notion databases into portals & apps with no code
Dev tools · 2024 · softr.io
One inbox for all your work discussions
Work · 2024 · generalcollaboration.com