Dev tools · alternatives · 2026
24 alternatives to All-pair similarity search on millions of sets in Python and on laptop
All-pair set similarity search on millions of sets in Python and on a laptop - GitHub - ekzhu/SetSimilaritySearch: All-pair set similarity search on millions of sets in Python and on a laptop
This is a Python developer tool for finding similar sets among millions of items, designed to run on standard laptops. It performs all-pair similarity searches efficiently without requiring specialized hardware or… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. All-pair similarity search on millions of sets in Python and on laptop launched in 2018; newer entries below may have overtaken it.
- 1
Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.
2019 · towardsdatascience.com · its alternatives →
- 2
I recently found myself computing the similarity between lots of very high dimensional vectors (i.e., sentence embedding vectors from LLMs), and I wanted to try some more powerful measures of similarity/dependency than just Cosine similarity, which seems to be the default for everything nowadays because of its computational efficiency. There are many other more involved measures that can detect more subtle relationships, but the problem is that some of them are quite slow to compute, especially if you're trying to do it in Python. For my favorite measure of statistical dependency,…
2023 · github.com · its alternatives →
- 3
:crystal_ball: Scalable reverse image search built on Kubernetes and Elasticsearch - dsys/match
2016 · github.com · its alternatives →
- 4
Hey HN! We (Stephan and Thomas) recently open-sourced Semble. We kept running into the same problem while using Claude Code on large codebases: when the agent can't find something directly, it falls back to grep, reading full files or launching subagents. This uses a lot of tokens, and often still misses the relevant code. There are existing tools for this, but they were either too slow to index on demand, needed API keys, or had poor retrieval quality. Semble is our solution for this. It combines static Model2Vec embeddings (using our latest static model: potion-code-16M) with BM25, fused…
May 2026 · github.com · its alternatives →
- 5
An on-disk B+tree for Python 3. Contribute to NicolasLM/bplustree development by creating an account on GitHub.
2018 · github.com · its alternatives →
- 6
- 7
Rapid fuzzy string matching in Python using various string metrics - rapidfuzz/RapidFuzz
2020 · github.com · its alternatives →
- 8DD
Gleb, Alex, Erez and Simon here – we are building an open-source tool for comparing data within and across databases at any scale. The repo is at https://github.com/datafold/data-diff, and our home page is https://datafold.com/. As a company, Datafold builds tools for data engineers to automate the most tedious and error-prone tasks falling through the cracks of the modern data stack, such as data testing and lineage. We launched two years ago with a tool for regression-testing changes to ETL code…
2022 · its alternatives →
- 9
HNSW implementation in Rust. Reference: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf - swapneel/hnsw-rust
2024 · github.com · its alternatives →
- 10
Pattern Matching for Python 3.7+ in a simple, yet powerful, extensible manner. - scravy/awesome-pattern-matching
2021 · github.com · its alternatives →
- 11
A container for dataclasses with multi-indexing and bulk operations. - joshlk/dataclassframe
2020 · github.com · its alternatives →
- 12
We’ve just open-sourced SemHash, a lightweight package for semantic text deduplication. It lets you effortlessly clean up your datasets and avoid pitfalls caused by duplicate samples in semantic search, RAG, and machine learning. Main Features: - Fast and hardware friendly: Deduplicate datasets with millions of records in minutes, on a CPU. - Flexible: Works on single or multiple datasets (e.g., train/test deduplication), and multi-column data (e.g., Question-Answering datasets). - Lightweight: Minimal dependencies (largest is NumPy). - Explainable: Easily inspect duplicates and what…
2025 · github.com · its alternatives →
- 13
Local first semantic and hybrid BM25 grep / search tool for use by AI and humans! - GitHub - BeaconBay/ck: Local first semantic and hybrid BM25 grep / search tool for use by AI and humans!
2025 · github.com · its alternatives →
- 14
We’ve just open-sourced Vicinity, a lightweight approximate nearest neighbors (ANN) search package that allows for fast experimentation and comparison of a larger number of well known algorithms. Main features: - Lightweight: the base package only uses Numpy - Unified interface: use any of the supported algorithms and backends with a single interface: HNSW, Annoy, FAISS, and many more algorithms and libraries are supported - Easy evaluation: evaluate the performance of your backend with a simple function to measure queries per second vs recall - Serialization: save and load your index for…
2024 · github.com · its alternatives →
- 15
Go library that provides fuzzy string matching optimized for filenames and code symbols in the style of Sublime Text, VSCode, IntelliJ IDEA et al. - sahilm/fuzzy
2018 · github.com · its alternatives →
- 16
I created a web page to compare different analytical databases (both self-managed and services, open-source and proprietary) on a realistic dataset. It contains 20+ databases, each with installation and data loading scripts. And they can be compared to each other on a set of 43 queries, by data load time or by storage size. There are switches to select different types of databases for comparison - for example, only MySQL compatible or PostgreSQL compatible. If you play with the switches, many interesting details will be uncovered. Full description:…
2022 · benchmark.clickhouse.com · its alternatives →
- 17
A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling in Python. - reiinakano/xcessiv
2017 · github.com · its alternatives →
- 18
A Python library for detecting patterns and anomalies in massive datasets using the Matrix Profile - target/matrixprofile-ts
2019 · github.com · its alternatives →
- 19
OctaneDB is an open-source vector database for Python that focuses on ultra-fast similarity search for high-dimensional data—perfect for AI/ML, semantic search, and large-scale document or embedding retrieval. What does it do? Store, index, and search millions of embeddings (text, images, etc.) with sub-millisecond query time. Supports in-memory and efficient HDF5 persistent storage. Integrates seamlessly with sentence-transformers for automatic text embedding. Key Features: 10x faster than Pinecone or ChromaDB for vector search and batch insertions. Advanced indexing: HNSW (approximate…
2025 · github.com · its alternatives →
- 20
Hey HN! I got nerd-sniped by Bloom Filters this weekend, specifically for searching datasets with high "cardinality" (number of unique items). They're an _amazing_ data structure that, at a fixed size, tracks potential set membership. That means unlike normal b-tree indexes, they don't grow with the number of unique items in the dataset. This makes them great for "needle in a haystack" search (logs, document) as implementations like VictoriaMetrics and Bing's BitFunnel show. I've used them in the past, but they've never been center-stage in my projects. I wanted high cardinality keyword…
2025 · github.com · its alternatives →
- 21
A tool for refurbishing and modernizing Python codebases - dosisod/refurb
2022 · github.com · its alternatives →
- 22
Advanced Bloom Filter Based Algorithms for Efficient Approximate Data De-Duplication in Streams - jparkie/PDD
2016 · github.com · its alternatives →
- 23
Hi everyone! I've been working on a web search interface for Python's documentation as a personal project, and I think it's ready for other people to use... Please give it a go (and join me in praying to the server gods): https://pythondocs.xyz/ Here's the tech stack for those interested: - Parser: Beautiful Soup + Mozilla Bleach - Database: in-memory SQLite (aiosqlite) + SQLAlchemy - Web server: FastAPI + Uvicorn + Jinja2 - Front end: Tailwind CSS + htmx + Alpine.js I have ideas for future improvements but hopefully the current version is useful to someone. Let me know what…
2022 · pythondocs.xyz · its alternatives →
- 24
Hey HN! I’ve recently open-sourced Pyversity, a lightweight library for diversifying retrieval results. Most retrieval systems optimize only for relevance, which can lead to top-k results that look almost identical. Pyversity efficiently re-ranks results to balance relevance and diversity, surfacing items that remain relevant but are less redundant. This helps with improving retrieval, recommendation, and RAG pipelines without adding latency or complexity. Main features: - Unified API: one function (diversify) supporting several well-known strategies: MMR, MSD, DPP, and COVER (with more to…
2025 · github.com · its alternatives →
Also compare
- How to analyse 100 GB of data on your laptop with Python alternatives
- Fast vector similarity using Rust and Python alternatives
- Scalable reverse image search built on Kubernetes and Elasticsearch alternatives
- Semble – Code search for agents that uses 98% fewer tokens than grep alternatives
- On-disk B+ tree for Python 3 alternatives
- Taipy 3.0 alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →