RAGstack – private ChatGPT for enterprise VPCs, built with Llama 2
Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information…
In plain words
RAGstack is an open-source platform that enables enterprises to deploy a ChatGPT-like application within their own virtual private cloud using Llama 2. It includes a vector database and API server, allowing organizations to upload files and connect the language model to proprietary data through retrieval augmented generation. The tool is designed for companies that need to keep sensitive information private while leveraging large language model capabilities for internal applications.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information from other systems and inserting them into the LLM’s context window via a prompt. This gives LLMs information beyond what was provided in their training data, which is necessary for almost every enterprise application. Examples include data from current web pages, data from SaaS apps like Confluence or Salesforce, and data from documents like sales contracts and PDFs. RAG works better than fine-tuning the model because it’s cheaper, it’s faster, and it’s more reliable since the provenance of information is attached to each response. While there are quite quite a few “chat with your data” apps at this point, most have external dependencies to APIs like OpenAI or Pinecone. RAGstack, on the other hand, only has open-source dependencies and lets you run the entire stack locally or on your cloud provider. This includes: - Containerizing LLMs like Falcon, Llama2, and GPT4all with Truss - Vector search with Qdrant. - File parsing and ingestion with Langchain, PyMuPDF, and Unstructured.io - Cloud deployment with Terraform If you want to dive into it yourself, we also published a couple of tutorials on how to deploy open source LLMs for your organization, and optionally give it access to internal documents without any data ever leaving your VPC. - How to deploy Llama 2 to Google Cloud (GCP): https://www.psychic.dev/post/how-to-deploy-llama-2-to-google... - How to connect Llama 2 to your own data using RAGstack: https://www.psychic.dev/post/how-to-self-host-llama-2-and-co... Let a thousand private corporate oracles bloom!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, July 2023
the whole month →
- WL
Hey everyone, I here is a small open-source project I've been working on lately. I'd love to hear your thoughts and improvement ideas :) GitHub: [github.com/Vincenius/workout-lol](https://github.com/Vincenius/workout-lol)
Dev tools · 2023 · workout.lol
- HN
I saw this [0] pretty cool thread by user revskill, and wanted a quicker way to search through it, but also to keep them all in one place so I can read them at my leisure whenever I get time. Right now is like 60 lines of Ruby using Nokogiri, but I will certainly look into it further down the line and improve the list. There's a cronjob checking the thread every 12 hours but I will eventually shut that down and it will become static after that. There are some really awesome blogs in there. I really recommend going through the list, it made my day. [0] "Could you share your personal blog…
Life & fun · 2023 · dm.hn
A safe space for your thoughts, private, local, p2p & open
Commerce · 2023 · anytype.io
