Open-source model and scorecard for measuring hallucinations in LLMs
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
Open-source model and scorecard for measuring hallucinations in LLMs launched on November 6, 2023 with 65 votes, #1,135 of 4,333 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
An open-source model and evaluation tool for detecting hallucinations in large language models, particularly in retrieval augmented generation systems. It identifies instances where LLMs fabricate details not present in source material during tasks like summarization. The tool includes an Apache 2.0 licensed model on HuggingFace, evaluation results comparing popular LLMs, and methodology documentation. Designed for developers and organizations building RAG systems who want to measure and improve hallucination rates in their models.
written from the facts on this page · September 2026
From the sources
Vectara launches open-source Hallucination Evaluation Model (HEM) that provides a FICO-like score for grading how often a generative LLM hallucinates in Retrieval Augmented Generation (RAG) systems. Today, we’re happy to announce the release of our open-source Hallucination Evaluation Model! One of the top concerns of enterprises considering adopting generative AI has been the possibility of LLMs generating hallucinations . Hallucinations come in many forms that could negatively affect a business: While many businesses acknowledge the benefits of generative AI, the risks of these hallucinations have held many back. Some attempts have been made in the past to quantify or at least qualify…from vectara.com
In the maker’s words, at launch
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some…
Does a similar job
all alternatives →
Llama 3.2 Interpretability with Sparse Autoencoders2024 · github.com · ▲579I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
OpenJuris – AI legal research with citations from primary sourcesFeb 2026 · openjuris.org · ▲18We built tooling that connects LLMs directly to case law databases with citation verification to address hallucination in legal AI. Think of it as giving the model access to actual legal sources instead of relying on training data.
- NMNomadic – Minimize RAG Hallucinations with 1 Hyperparameter Experiment2024 · ▲95
Hey HN! Mustafa, Lizzie, and Varun here from NomadicML (https://nomadicml.com). We’re excited to show you Nomadic (https://github.com/nomadic-ml/nomadic): a platform focused on parameter search to continuously optimize AI systems. Here’s a simple demo notebook where you get the best-performing, statistically significant configurations for your RAG — and improve hallucination metrics by 4X in just 5 minutes — with a single Nomadic experiment: https://tinyurl.com/4xmaryyw Our lightweight library is now live on PyPI (`pip install nomadic`). Try one…
Sup AIApr 2026 · sup.ai · ▲103Hi HN. I'm Ken, a 20-year-old Stanford CS student. I built Sup AI. I started working on this because no single AI model is right all the time, but their errors don’t strongly correlate. In other words, models often make unique mistakes relative to other models. So I run multiple models in parallel and synthesize the outputs by weighting segments based on confidence. Low entropy in the output token probability distributions correlates with accuracy. High entropy is often where hallucinations begin. My dad Scott (AI Research Scientist at TRI) is my research partner on this. He sends me papers…
- FAFactual AI Q&A – Answers based on Huberman Lab transcripts2022 · huberman.rile.yt · ▲120
This is a quick prototype I built for semantic search and factual question answering using embeddings and GPT-3. It tries to solve the LLM hallucination issue by guiding it only to answer questions from the given context instead of making things up. If you ask something not covered in an episode, it should say that it doesn't know rather than providing a plausible, but potentially incorrect response. It uses Whisper to transcribe, text-embedding-ada-002 to embed, Pinecone.io to search, and text-davinci-003 to generate the answer. More examples and explanations here:…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, November 2023
the whole month →
Discover & book top creators to promote your product
Growth · 2023 · passionfroot.me
SpeedVitals▲829Hi HN, I’m Kashish, one of the co-founders of SpeedVitals, a platform for testing & monitoring website performance with a major emphasis on Core Web Vitals. Until now, our platform has only focused on Synthetic/Lab testing, such as Lighthouse tests and TTFB tests. But now, we have developed a Real-User Monitoring platform that doubles up as a Web Analytics tool. One major feature we have that most RUM tools lack is Performance Audits. Just like Lighthouse audits work on Lab test data, our audits work across all the real-user visits and gives suggestions on what task to prioritize first.…
Dev tools · 2023 · speedvitals.com