nowfound

Dev tools · August 23, 2023

Fast vector similarity using Rust and Python

I recently found myself computing the similarity between lots of very high dimensional vectors (i.e., sentence embedding vectors from LLMs), and I wanted to try some more powerful measures of similarity/dependency than just Cosine similarity, which seems to be the default for everything nowadays because of its computational efficiency. There are many other more involved measures that can detect more subtle relationships, but the problem is that some of them are quite slow to compute, especially if you're trying to do it in Python. For my favorite measure of statistical dependency,…

Fast vector similarity using Rust and Python launched on August 23, 2023 with 141 votes, #366 of 4,347 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 3 years on.

Visit github.com ↗Alternativestop 8% of August 2023

In plain words

Fast Vector Similarity is a library that computes similarity and statistical dependency measures between high-dimensional vectors using Rust and Python. It offers alternatives to cosine similarity, including measures like Hoeffding's D, which can detect more subtle relationships in data such as sentence embeddings from language models. The tool leverages Rust's performance to make computationally intensive similarity calculations faster in Python than traditional NumPy implementations.

written from the facts on this page · September 2026

From the sources

High-performance vector similarity library in Rust with Python bindings: Spearman, Kendall, distance correlation, Jensen-Shannon, Hoeffding's D, and bootstrapped confidence intervals - Dicklesworthstone/fast_vector_similarity

In the maker’s words, at launch

I recently found myself computing the similarity between lots of very high dimensional vectors (i.e., sentence embedding vectors from LLMs), and I wanted to try some more powerful measures of similarity/dependency than just Cosine similarity, which seems to be the default for everything nowadays because of its…

Does a similar job

all alternatives →
  • A fast HNSW implementation in Rust2024 · github.com · ▲111

    HNSW implementation in Rust. Reference: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf - swapneel/hnsw-rust

  • All-pair similarity search on millions of sets in Python and on laptop2018 · github.com · ▲123

    All-pair set similarity search on millions of sets in Python and on a laptop - GitHub - ekzhu/SetSimilaritySearch: All-pair set similarity search on millions of sets in Python and on a laptop

  • IM
    I made a website to semantically search ArXiv papers2024 · papermatch.mitanshu.tech · ▲324

    As a grad student (and an ADHDer), I had trouble doing literature review systematically. To combat this, I made a website that finds similar papers using the meaning of the thing I am looking for. I used MixedBread's [^1] embedding model to generate vectors from the abstracts. I store and search similar vectors using Milvus [^2] and finally use Gradio [^3] to serve the frontend. I update the vector database weekly by pulling the metadata dataset from Kaggle [^4]. To speed up the search process on my free oracle instance, I binarise the embeddings and use Hamming distance as a metric. I would…

  • Vector – A High-Performance Log and Metric Router Written in Rust2019 · github.com · ▲322

    A high-performance observability data pipeline. Contribute to vectordotdev/vector development by creating an account on GitHub.

  • Word2vec Algorithm in ~100sloc with NumPy2023 · github.com · ▲72

    Here's a small demonstration of the fundamental aspects of the word-to-vec algorithm. It's implemented in a single python script and depends only on a single text file for training. It's not meant to be blazingly fast or anything, just a toy example to aid my understanding of how word vectors might be learnt from a corpus.

  • High-Performance Wavelet Matrix for Python, Implemented in RustDec 2025 · pypi.org · ▲93

    I built a Rust-powered Wavelet Matrix library for Python. There were surprisingly few practical Wavelet Matrix implementations available for Python, so I implemented one with a focus on performance, usability, and typed APIs. It supports fast rank/select, top-k, quantile, range queries, and even dynamic updates. Feedback welcome!

More dev tools this month

the category →
  • /dev/null▲472

    a social network for devs

    Dev tools · 2d ago · dev-null.sh

  • open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

    Dev tools · 27d ago · github.com

  • Contribute to reladraw/reladraw development by creating an account on GitHub.

    Dev tools · 25d ago · github.com

  • OpenBot▲362

    Grok Bot alternative: free, local, open-source, multiplayer

    Dev tools · 22d ago · openbot.run

  • Talanoa▲340

    Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

    Dev tools · 20d ago · talanoa.email

  • CoreSpeed▲332

    One MCP for everything your agents need: apps, memory, tools

    Dev tools · 27d ago · corespeed.io

Source: Show HN post ↗ · launch text quoted in part

Launched alongside, August 2023

the whole month →
  • Resend▲1,325

    An entire collection of pre-built components with a much faster development experience.

    Dev tools · 2023 · resend.com

  • STORI AI▲1,316

    Your ideas become visually compelling branded social posts

    AI · 2023 · storiai.com

  • Kombai▲1,079

    A new AI model that you can prompt with UI designs

    AI · 2023 · kombai.com

  • Understand your audience without 50 interviews

    AI · 2023 · founderpal.ai

  • Lottielab▲869

    Create and ship lottie animations to sites and apps faster

    Dev tools · 2023 · lottielab.com

  • Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…

    AI · 2023 · github.com