AI · alternatives · 2026

24 alternatives to Open-RAG-Eval

RAG Evaluation that works!

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
Ragas – Open-source library for evaluating RAG pipelines▲121
Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/. Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year. Problems Ragas can solve - How do you choose the best components for your RAG, such as the retriever, reranker, and LLM? - How do you formulate a test dataset…
2024 · github.com · its alternatives →
2
Tonic Validate Metrics – an open-source RAG evaluation metrics package▲40
Hey HN, Joe and Ethan from Tonic.ai here. We just released a new open-source python package for evaluating the performance of Retrieval Augmented Generation (RAG) systems. Earlier this year, we started developing a RAG-powered app to enable companies to talk to their free-text data safely. During our experimentation, however, we realized that using such a new method meant that there weren’t industry-standards for evaluation metrics to measure the accuracy of RAG performance. We built Tonic Validate Metrics (tvalmetrics, for short) to easily calculate the benchmarks we needed to meet in…
2023 · github.com · its alternatives →
3
Ragas – Open-source library for evals and testing RAG systems▲15
Ragas is an open-source library designed for evaluating and testing RAG (Retrieval-Augmented Generation) and other LLM applications. It offers a diverse set of metrics and methods, including synthetic test data generation, to help you assess your RAG applications. Ragas was initially developed to address our own needs for evaluating RAG chatbots last year. ### Problems Ragas Can Solve: - How can you select the best components for your RAG, such as the retriever, reranker, and LLM? - How can you create a test dataset without incurring significant expenses and time? We believe there's a need…
2024 · github.com · its alternatives →
4
Supavec▲670
The open source RAG as a service platform
2025 · supavec.com · its alternatives →
5
Super RAG▲114
Super performant RAG pipelines for AI apps
2024 · github.com · its alternatives →
6
FastGraphRAG – Better RAG using good old PageRank▲457
Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
2024 · github.com · its alternatives →
7
Open-source Rule-based PDF parser for RAG▲293
The PDF parser is a rule based parser which uses text co-ordinates (boundary box), graphics and font data. The PDF parser works off text layer and also offers a OCR option to automatically use OCR if there are scanned pages in your PDFs. The OCR feature is based off a modified version of tika which uses tesseract underneath. The PDF Parser offers the following features: * Sections and subsections along with their levels. * Paragraphs - combines lines. * Links between sections and paragraphs. * Tables along with the section the tables are found in. * Lists and nested lists. * Join content…
2024 · github.com · its alternatives →
8
Cognita – open-source RAG framework for modular applications▲142
Hey HN, exciting news! Our RAG framework, Cognita (https://github.com/truefoundry/cognita), born from collaborations with diverse enterprises, is now open-source. Currently, it offers seamless integrations with Qdrant and SingleStore. In recent weeks, numerous engineers have explored Cognita, providing invaluable insights and feedback. We deeply appreciate your input and encourage ongoing dialogue (share your thoughts in the comments – let's keep this ‘open source’). While RAG is undoubtedly powerful, the process of building a functional application with it can feel…
2024 · github.com · its alternatives →
9
R2R – Open-source framework for production-grade RAG▲167
Hello HN, I'm Owen from SciPhi (https://www.sciphi.ai/), a startup working on simplifying˛Retrieval-Augmented Generation (RAG). Today we’re excited to share R2R (https://github.com/SciPhi-AI/R2R), an open-source framework that makes it simpler to develop and deploy production-grade RAG systems. Just a quick reminder: RAG helps Large Language Models (LLMs) use current information and specific knowledge. For example, it allows a programming assistant to use your latest documents to answer questions. The idea is to gather all the relevant information…
2024 · github.com · its alternatives →
10
Doks▲41
RAG Creation Tool
2025 · its alternatives →
11
Evaluate LLM-based RAG Applications with automated test set generation▲13
🐢 Open-Source Evaluation & Testing library for LLM Agents - Giskard-AI/giskard-oss
2024 · github.com · its alternatives →
12
Rag About It▲48
Dive deep into AI Retrieval Augmented Generation (RAG)
2024 · ragaboutit.com · its alternatives →
13
Morphik – Open-source RAG that understands PDF images, runs locally▲200
Hey HN, we’re Adi and Arnav. A few months ago, we hit a wall trying to get LLMs to answer questions over research papers and instruction manuals. Everything worked fine, until the answer lived inside an image or diagram embedded in the PDF. Even GPT‑4o flubbed it (we recently tried O3 with the same, and surprisingly it flubbed it too). Naive RAG pipelines just pulled in some text chunks and ignored the rest. We took an invention disclosure PDF (https://drive.google.com/file/d/1ySzQgbNZkC5dPLtE3pnnVL2rW_9...) containing an IRR‑vs‑frequency graph and asked GPT “From…
2025 · github.com · its alternatives →
14
Open-Source Colab Notebooks to Implement Advanced RAG Techniques▲98
Hey HN fam, We’ve seen developers spend a lot of time implementing advanced RAG techniques from scratch. While these techniques are essential for improving performance, their implementation requires a lot of effort and testing! To help with this process, our team (Athina AI) has released Open-Source Advanced RAG Cookbooks. This is a collection of ready-to-run Google Colab notebooks featuring the most commonly implemented techniques. Please show us some love by starring the repo if you find this useful!
2024 · github.com · its alternatives →
15
LiquidIndex 2.0▲134
I'm building LiquidIndex, an RAG API service that makes it easy to add personalized context to AI apps. Instead of building a whole system to upload, process, and search through a user’s files or notes, you just call a few APIs. It handles the hard parts - like connecting to Notion or Google Drive, breaking the data into usable pieces, and making it searchable. Then, when you want to answer a question using that data, you just query it, and it gives you the most relevant results - optionally with an LLM-generated answer. No pipelines, no infrastructure, no headaches. Here are the core APIs:…
2025 · liquidindex.dev · its alternatives →
16
Rag Document Viewer▲22
High‑Fidelity Document Viewer for RAG Apps
2025 · github.com · its alternatives →
17
RAGstack▲111
Deploy a private ChatGPT alternative hosted within your VPC
2023 · github.com · its alternatives →
18
RAGBuilder – Hyperparameter tuning on various RAG parameters▲17
A RAG has several moving parts: data ingestion, retrieval, re-ranking, generation etc.. Each part comes with numerous options. If we consider a toy example, where you could choose from: 5 different chunking methods, 5 different chunk sizes, 5 different embedding models, 5 different retrievers, 5 different re-rankers/ compressors 5 different prompts 5 different LLMs That’s 78,125 distinct RAG configurations! If you could try evaluating each one in just 5 mins, that’d still take 271 days of non-stop trial-and-error effort! In short, it’s kinda impossible to find your optimal RAG setup…
2024 · github.com · its alternatives →
19
OpenFable – Open-source RAG engine using tree-structured indexes▲5
Hi HN, I built OpenFable, an open-source retrieval engine that implements the FABLE algorithm (https://arxiv.org/abs/2601.18116) for RAG pipelines. I'm using it in another project and thought that others might benefit. Most RAG systems chunk documents into flat segments and retrieve by vector similarity. This works for simple lookups but breaks when answers span multiple sections, when relevant content is buried in a subsection, or when you need to control how many tokens you're sending to an LLM. OpenFable takes a different approach: when you ingest a document, it uses…
Apr 2026 · github.com · its alternatives →
20
Opik, an open source LLM evaluation framework▲86
Hey HN! I'm Caleb, one of the contributors to Opik, a new open source framework for LLM evaluations. Over the last few months, my colleagues and I have been working on a project to solve what we see as the most painful parts of writing evals for an LLM application. For this initial release, we've focused on a few core features that we think are the most essential: - Simplifying the implementation of more complex LLM-based evaluation metrics, like Hallucination and Moderation. - Enabling step-by-step tracking, such that you can test and debug each individual component of your LLM application,…
2024 · github.com · its alternatives →
21
Open-source model and scorecard for measuring hallucinations in LLMs▲65
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com · its alternatives →
22
AutoRAG – AutoML tool for RAG. Find an optimal RAG pipeline▲18
- Discovering the most effective RAG pipeline for your specific data and use case can be daunting. It requires experimenting with various RAG modules and configurations, which are both time-consuming and complex. - AutoRAG addresses this challenge by automatically evaluating different combinations of RAG modules and their parameters. You don't need to write implementation code yourself; everything is set up through a single YAML file. - Our aim is to save you the hassle of continuously adapting to new RAG modules and configurations. Instead, you can focus on developing robust data for your…
2024 · github.com · its alternatives →
23
RAG-chunk – A CLI to test RAG chunking strategies▲10
A Python CLI to test, benchmark, and find the best RAG chunking strategy for your Markdown documents. - messkan/rag-chunk
Nov 2025 · github.com · its alternatives →
24
An extensive set of RAG implementations+many different strategies▲6
Hi all, Sharing a repo I was working on for a while. It’s open-source and includes many different strategies for RAG (currently 17), including tutorials, and visualizations. This is great learning and reference material. Open issues, suggest more strategies, and use as needed. Enjoy!
2024 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →