AI · alternatives · 2026

24 alternatives to Ragas – Open-source library for evaluating RAG pipelines

Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/. Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year. Problems Ragas can solve - How do you choose the best components for your RAG, such as the retriever, reranker, and LLM? - How do you formulate a test dataset…

Ragas is an open-source library for evaluating retrieval-augmented generation (RAG) and other large language model applications. It provides metrics and methods including synthetic test data generation to help… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Ragas – Open-source library for evaluating RAG pipelines launched in 2024; newer entries below may have overtaken it.

1
Ragas – Open-source library for evals and testing RAG systems▲15
Ragas is an open-source library designed for evaluating and testing RAG (Retrieval-Augmented Generation) and other LLM applications. It offers a diverse set of metrics and methods, including synthetic test data generation, to help you assess your RAG applications. Ragas was initially developed to address our own needs for evaluating RAG chatbots last year. ### Problems Ragas Can Solve: - How can you select the best components for your RAG, such as the retriever, reranker, and LLM? - How can you create a test dataset without incurring significant expenses and time? We believe there's a need…
2024 · github.com · its alternatives →
2
Demystifying Advanced RAG Pipelines▲131
I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
2023 · github.com · its alternatives →
3
FastGraphRAG – Better RAG using good old PageRank▲457
Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
2024 · github.com · its alternatives →
4
RAGstack▲111
Deploy a private ChatGPT alternative hosted within your VPC
2023 · github.com · its alternatives →
5
Turn any website into a knowledge base for LLMs▲305
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
2024 · embedding.io · its alternatives →
6
Supavec▲670
The open source RAG as a service platform
2025 · supavec.com · its alternatives →
7
RAGaaS▲75
The API for building production-ready AI apps with your data
2025 · sitegpt.ai · its alternatives →
8
R2R – Open-source framework for production-grade RAG▲167
Hello HN, I'm Owen from SciPhi (https://www.sciphi.ai/), a startup working on simplifying˛Retrieval-Augmented Generation (RAG). Today we’re excited to share R2R (https://github.com/SciPhi-AI/R2R), an open-source framework that makes it simpler to develop and deploy production-grade RAG systems. Just a quick reminder: RAG helps Large Language Models (LLMs) use current information and specific knowledge. For example, it allows a programming assistant to use your latest documents to answer questions. The idea is to gather all the relevant information…
2024 · github.com · its alternatives →
9
Tonic Validate Metrics – an open-source RAG evaluation metrics package▲40
Hey HN, Joe and Ethan from Tonic.ai here. We just released a new open-source python package for evaluating the performance of Retrieval Augmented Generation (RAG) systems. Earlier this year, we started developing a RAG-powered app to enable companies to talk to their free-text data safely. During our experimentation, however, we realized that using such a new method meant that there weren’t industry-standards for evaluation metrics to measure the accuracy of RAG performance. We built Tonic Validate Metrics (tvalmetrics, for short) to easily calculate the benchmarks we needed to meet in…
2023 · github.com · its alternatives →
10
Tilores Identity RAG▲323
Customer data search, unification and retrieval for LLMs
2024 · tilores.io · its alternatives →
11
Super RAG▲114
Super performant RAG pipelines for AI apps
2024 · github.com · its alternatives →
12
Open-Source Colab Notebooks to Implement Advanced RAG Techniques▲98
Hey HN fam, We’ve seen developers spend a lot of time implementing advanced RAG techniques from scratch. While these techniques are essential for improving performance, their implementation requires a lot of effort and testing! To help with this process, our team (Athina AI) has released Open-Source Advanced RAG Cookbooks. This is a collection of ready-to-run Google Colab notebooks featuring the most commonly implemented techniques. Please show us some love by starring the repo if you find this useful!
2024 · github.com · its alternatives →
13
Cognita – open-source RAG framework for modular applications▲142
Hey HN, exciting news! Our RAG framework, Cognita (https://github.com/truefoundry/cognita), born from collaborations with diverse enterprises, is now open-source. Currently, it offers seamless integrations with Qdrant and SingleStore. In recent weeks, numerous engineers have explored Cognita, providing invaluable insights and feedback. We deeply appreciate your input and encourage ongoing dialogue (share your thoughts in the comments – let's keep this ‘open source’). While RAG is undoubtedly powerful, the process of building a functional application with it can feel…
2024 · github.com · its alternatives →
14
Evaluate LLM-based RAG Applications with automated test set generation▲13
🐢 Open-Source Evaluation & Testing library for LLM Agents - Giskard-AI/giskard-oss
2024 · github.com · its alternatives →
15
Vectorize▲246
Build RAG pipelines that are optimized for your data.
2024 · vectorize.io · its alternatives →
16
RAGatouille, a simple lib to use&train top retrieval models in RAG apps▲15
Hey HN! If you’re at all interested in LLMs/LLM-apps, you’ve probably heard of RAG: Retrieval-Assisted Generation, i.e. retrieving relevant documents to give to your LLM as context to answer user queries. Today, I’m releasing RAGatouille v0.0.1, whose aim is to make it as easy as can be to improve your RAG pipelines by leveraging state-of-the-art Information Retrieval research. As of right now, there’s quite a big gap between common everyday practice and the IR literature, and a lot of the gap is because there just aren’t good ways to quickly try out and leverage SotA IR techniques.…
2024 · github.com · its alternatives →
17
PromptTools – open-source tools for evaluating LLMs and vector DBs▲211
Hey HN! We’re Kevin and Steve. We’re building PromptTools (https://github.com/hegelai/prompttools): open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. Evaluating prompts, LLMs, and vector databases is a painful, time-consuming but necessary part of the product engineering process. Our tools allow engineers to do this in a lot less time. By “evaluating” we mean checking the quality of a model's response for a given use case, which is a combination of testing and benchmarking. As examples: - For generated…
2023 · github.com · its alternatives →
18
Dera – A platform to manage chunks and embeddings for building RAG apps▲25
Hi everyone, my cofounder and I built Dera - a platform to help manage chunks and embeddings. We built this because of the pain points we experienced while building RAG applications for side projects. The biggest pain point we encountered was that we were constantly trying out different chunking strategies, but there’s no easy way to check how the strategies are performing in terms of retrieval when given the same query. We tried searching for a tool for this but couldn’t find any (most LLM dev tools focus on prompts management). We hope this tool will be useful for people building RAG apps.…
2024 · getdera.com · its alternatives →
19
Verta RAG System▲84
Go from dataset to custom RAG prototype in 5 minutes
2024 · cloudera.com · its alternatives →
20
R2R V2 – A open source RAG engine with prod features▲251
Hi HN! We're building R2R [https://github.com/SciPhi-AI/R2R], an open source RAG answer engine that is built on top of Postgres+Neo4j. The best way to get started is with the docs - https://r2r-docs.sciphi.ai/introduction. This is a major update from our V1 which we have spent the last 3 months intensely building after getting a ton of great feedback from our first Show HN (https://news.ycombinator.com/item?id=39510874). We changed our focus to building a RAG engine instead of a framework, because this is what developers asked for the most.…
2024 · github.com · its alternatives →
21
RAGLite – A Python package for the unhobbling of RAG▲19
RAGLite is a Python package for building Retrieval-Augmented Generation (RAG) applications. RAG applications can be magical when they work well, but anyone who has built one knows how much the output quality depends on the quality of retrieval and augmentation. With RAGLite, we set out to unhobble RAG by mapping out all of its subproblems and implementing the best solutions to those subproblems. For example, RAGLite solves the chunking problem by partitioning documents in provably optimal level 4 semantic chunks. Another unique contribution is its optimal closed-form linear query adapter…
2024 · github.com · its alternatives →
22
Rag About It▲48
Dive deep into AI Retrieval Augmented Generation (RAG)
2024 · ragaboutit.com · its alternatives →
23
Open-source model and scorecard for measuring hallucinations in LLMs▲65
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com · its alternatives →
24
Dabarqus▲92
Add PDF chat to your LLM app in less than 9 lines of code
2024 · dabarqus.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →