Pyversity – Fast Result Diversification for Retrieval and RAG
Hey HN! I’ve recently open-sourced Pyversity, a lightweight library for diversifying retrieval results. Most retrieval systems optimize only for relevance, which can lead to top-k results that look almost identical. Pyversity efficiently re-ranks results to balance relevance and diversity, surfacing items that remain relevant but are less redundant. This helps with improving retrieval, recommendation, and RAG pipelines without adding latency or complexity. Main features: - Unified API: one function (diversify) supporting several well-known strategies: MMR, MSD, DPP, and COVER (with more to…
Pyversity – Fast Result Diversification for Retrieval and RAG launched on October 19, 2025 with 86 votes, #338 of 8,178 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
Pyversity is a lightweight open-source library that re-ranks retrieval results to reduce redundancy while maintaining relevance. It addresses the problem where most retrieval systems return similar top results by offering multiple diversification strategies including MMR, MSD, DPP, and COVER through a single unified API. Designed for retrieval, recommendation, and RAG pipelines, it requires only NumPy as a dependency and processes results in milliseconds, providing an efficient alternative to expensive cross-encoder re-ranking methods.
written from the facts on this page · September 2026
From the sources
Fast Diversification for Search & Retrieval. Contribute to Pringled/pyversity development by creating an account on GitHub.
Pyversity is a fast, lightweight library for diversifying retrieval results. Retrieval systems often return highly similar items. Pyversity efficiently re-ranks these results to encourage diversity, surfacing items that remain relevant but less redundant. It implements several popular diversification strategies such as MMR, MSD, DPP, SSD, and Cover with a clear, unified API and benchmarks for each strategy. More information about the supported strategies can be found in the supported strategies section . The only dependency is NumPy, making the package very lightweight. import numpy as np from pyversity import diversify , Strategy # Define embeddings and scores (e.g. cosine similarities of…from github.com
In the maker’s words, at launch
Hey HN! I’ve recently open-sourced Pyversity, a lightweight library for diversifying retrieval results. Most retrieval systems optimize only for relevance, which can lead to top-k results that look almost identical. Pyversity efficiently re-ranks results to balance relevance and diversity, surfacing items that remain…
Does a similar job
all alternatives →
FastGraphRAG – Better RAG using good old PageRank2024 · github.com · ▲457Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
Pygooglenews – Python library for advanced Google News data mining2020 · github.com · ▲165If Google News had a Python library. Contribute to kotartemiy/pygooglenews development by creating an account on GitHub.
uFuzzy.js – A tiny, efficient fuzzy search that doesn't suck2022 · github.com · ▲251Hello HN! I became frustrated with the unpredictible/poor match quality and opaqueness of "relevance scores" in existing fuzzy and fulltext search libs, so I tried something different and this is the result. The main selling point is the result quality / ordering, with best-in-class memory overhead and excellent performance being bonuses. The API is pretty stable at this point, but looking for feedback before committing to 1.0. TL;DR The test corpus is a 4MB json file with 162k words/phrases, so give it a second for initial download. You can also drag/drop your own…
Scalable reverse image search built on Kubernetes and Elasticsearch2016 · github.com · ▲139:crystal_ball: Scalable reverse image search built on Kubernetes and Elasticsearch - dsys/match
Demystifying Advanced RAG Pipelines2023 · github.com · ▲131I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, October 2025
the whole month →
I went down the rabbit hole on a side project and ended up building this: Strange Attractors(https://blog.shashanktomar.com/posts/strange-attractors). It’s built with three.js. Working on it reminded me of the little "maths for fun" exercises I used to do while learning programming in early days. Just trying things out, getting fascinated and geeky, and being surprised by the results. I spent way too much time on this, but it was extreme fun. My favorite part: someone pointed me to the Simone Attractor on Threads. It is a 2D attractor and I asked GPT to extrapolate it to…
AI · Oct 2025 · blog.shashanktomar.com
Autism Simulator▲779Hey all, I built this. It’s not trying to capture every autistic experience (that’d be impossible). It’s based on my own lived experience as well as that of friends on the spectrum. I'm trying to give people a feel for what masking, decision fatigue, and burnout can look like day-to-day. That’s hard to explain in words, but easier to show through choices and stats. I'm not trying to "define autism". I’ve gotten good feedback here about resilience, meds, and difficulty tuning. I’ll keep tweaking it. If even a few people walk away thinking, "ah, maybe that’s why my coworker struggles in those…
Life & fun · 2025 · autism-simulator.vercel.app
