Tonic Validate Metrics – an open-source RAG evaluation metrics package
Hey HN, Joe and Ethan from Tonic.ai here. We just released a new open-source python package for evaluating the performance of Retrieval Augmented Generation (RAG) systems. Earlier this year, we started developing a RAG-powered app to enable companies to talk to their free-text data safely. During our experimentation, however, we realized that using such a new method meant that there weren’t industry-standards for evaluation metrics to measure the accuracy of RAG performance. We built Tonic Validate Metrics (tvalmetrics, for short) to easily calculate the benchmarks we needed to meet in…
Tonic Validate Metrics – an open-source RAG evaluation metrics package launched on October 25, 2023 with 40 votes, #1,262 of 4,393 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Tonic Validate Metrics is an open-source Python package for evaluating Retrieval Augmented Generation (RAG) systems. Built by Tonic.ai, it provides standardized metrics to measure RAG accuracy and performance, addressing the lack of established evaluation benchmarks in the field. The package is designed for developers and companies building RAG-powered applications that need to safely enable users to query free-text data.
written from the facts on this page · September 2026
From the sources
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications. - TonicAI/tonic_validate
In the maker’s words, at launch
Hey HN, Joe and Ethan from Tonic.ai here. We just released a new open-source python package for evaluating the performance of Retrieval Augmented Generation (RAG) systems. Earlier this year, we started developing a RAG-powered app to enable companies to talk to their free-text data safely. During our experimentation…
Does a similar job
all alternatives →
Ragas – Open-source library for evaluating RAG pipelines2024 · github.com · ▲121Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/. Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year. Problems Ragas can solve - How do you choose the best components for your RAG, such as the retriever, reranker, and LLM? - How do you formulate a test dataset…
FastGraphRAG – Better RAG using good old PageRank2024 · github.com · ▲457Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
RAGLite – A Python package for the unhobbling of RAG2024 · github.com · ▲19RAGLite is a Python package for building Retrieval-Augmented Generation (RAG) applications. RAG applications can be magical when they work well, but anyone who has built one knows how much the output quality depends on the quality of retrieval and augmentation. With RAGLite, we set out to unhobble RAG by mapping out all of its subproblems and implementing the best solutions to those subproblems. For example, RAGLite solves the chunking problem by partitioning documents in provably optimal level 4 semantic chunks. Another unique contribution is its optimal closed-form linear query adapter…
Cognita – open-source RAG framework for modular applications2024 · github.com · ▲142Hey HN, exciting news! Our RAG framework, Cognita (https://github.com/truefoundry/cognita), born from collaborations with diverse enterprises, is now open-source. Currently, it offers seamless integrations with Qdrant and SingleStore. In recent weeks, numerous engineers have explored Cognita, providing invaluable insights and feedback. We deeply appreciate your input and encourage ongoing dialogue (share your thoughts in the comments – let's keep this ‘open source’). While RAG is undoubtedly powerful, the process of building a functional application with it can feel…
R2R – Open-source framework for production-grade RAG2024 · github.com · ▲167Hello HN, I'm Owen from SciPhi (https://www.sciphi.ai/), a startup working on simplifying˛Retrieval-Augmented Generation (RAG). Today we’re excited to share R2R (https://github.com/SciPhi-AI/R2R), an open-source framework that makes it simpler to develop and deploy production-grade RAG systems. Just a quick reminder: RAG helps Large Language Models (LLMs) use current information and specific knowledge. For example, it allows a programming assistant to use your latest documents to answer questions. The idea is to gather all the relevant information…
Ragas – Open-source library for evals and testing RAG systems2024 · github.com · ▲15Ragas is an open-source library designed for evaluating and testing RAG (Retrieval-Augmented Generation) and other LLM applications. It offers a diverse set of metrics and methods, including synthetic test data generation, to help you assess your RAG applications. Ragas was initially developed to address our own needs for evaluating RAG chatbots last year. ### Problems Ragas Can Solve: - How can you select the best components for your RAG, such as the retriever, reranker, and LLM? - How can you create a test dataset without incurring significant expenses and time? We believe there's a need…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, October 2023
the whole month →
Nudge 2.0▲1,051In-app experiences to activate, retain, & understand users
Growth · 2023 · nudgenow.com
Unlock AI magic for elevated customer engagement, fast
AI · 2023 · tiledesk.com
Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website.
Dev tools · 2023 · github.com