AI · alternatives · 2026

24 alternatives to Trieve Vector Inference

Deploy fast, unmetered embedding inference in your own VPC

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Trieve Vector Inference launched in 2024; newer entries below may have overtaken it.

1
Speeding up LLM inference 2x times (possibly)▲419
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
2
General Compute▲315
AI models that run on an inference cloud optimized for speed
May 2026 · generalcompute.com · its alternatives →
3
Supabase Vector▲398
The open source backend for AI applications
2023 · supabase.com · its alternatives →
4
Pip install inference, open source computer vision deployment▲71
Deploying vision models is time consuming and tedious. Setting up dependencies. Fixing conflicts. Configuring TRT acceleration. Flashing (and re-flashing) NVIDIA Jetsons. A streamlined, developer-friendly solution for inference is needed. We, the Roboflow team, have been hard at work open sourcing Inference, an open source vision deployment solution. Our solution is designed with developers in mind, offering a HTTP-based interface. Run models on your hardware without having to write architecture-specific inference code. Here's a demo showing how to go from a model to GPU inference on a video…
2023 · github.com · its alternatives →
5
Prompts as WASM Programs▲209
AICI is a proposed common interface between LLM inference engines (llama.cpp, vLLM, HF Transformers, etc.) and "controllers" - programs that can constrain the LLM output according to regexp, grammar, or custom logic, as well as control the generation process (forking, backtracking, etc.). AICI is based on Wasm, and is designed to be fast (runs on CPU while GPU is busy), secure (can run in multi-tenant cloud deployments), and flexible (allow libraries like Guidance, LMQL, Outlines, etc. to work on top of it). We (Microsoft Research) have released it recently, and would love feedback on the…
2024 · github.com · its alternatives →
6
Faster FastAPI with simdjson and io_uring on Linux 5.19▲290
A few months ago, I benchmarked FastAPI on an i9 MacBook Pro. I couldn't believe my eyes. A primary REST endpoint to `sum` two integers took 6 milliseconds to evaluate. It is okay if you are targeting a server in another city, but it should be less when your client and server apps are running on the same machine. FastAPI would have bottleneck-ed the inference of our lightweight UForm neural networks recently trending on HN under the title "Beating OpenAI CLIP with 100x less data and compute". (Thank you all for the kind words!) So I wrote another library. It has been a while since I have…
2023 · github.com · its alternatives →
7
ZeroGPU▲298
The compute efficient layer for AI inference
Jun 2026 · zerogpu.ai · its alternatives →
8
Inference Engine by GMI Cloud▲180
Fast multimodal-native inference at scale
Dec 2025 · gmicloud.ai · its alternatives →
9
Thinc, a new deep learning library by the makers of spaCy and FastAPI▲250
Thinc is a lightweight type-checked deep learning library for composing models, with support for layers defined in frameworks like PyTorch and TensorFlow.
2020 · thinc.ai · its alternatives →
10
Tiny-vLLM – high performance LLM inference engine in C++ and CUDA▲205
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
11
Synapsis - Vectorisation as a Service▲30
We connect visual data to AI applications.
2024 · quasara.io · its alternatives →
12
I built a tensor library from scratch in C++/CUDA▲119
Hi HN, Over the past few months, I've been building `dsc`, a tensor library from scratch in C++/CUDA. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally. The key features are: - C++ core with CUDA support written from scratch. - A familiar, PyTorch-like Python API. - Runs real models: it's complete enough to load a model like Qwen from HuggingFace and run inference on both CUDA and CPU with a single line change[1]. - Simple, built-in observability for both Python and C++. Next on the roadmap is…
2025 · github.com · its alternatives →
13AG
A GPU-accelerated binary vector index▲65
This is a vector index I built that supports insertion and k-nearest neighbors (k-NN) querying, optimized for GPUs. It operates entirely in CUDA and can process queries on half a billion vectors in under 200 milliseconds. The codebase is structured as a standalone library with an HTTP API for remote access. It’s intended for high-performance search tasks—think similarity search, AI model retrieval, or reinforcement learning replay buffers. The codebase is located at https://github.com/rodlaf/BinaryGPUIndex.
2025 · rlafuente.com · its alternatives →
14
I replaced vector databases with Git for AI memory (PoC)▲198
Hey HN! I built a proof-of-concept for AI memory using Git instead of vector databases. The insight: Git already solved versioned document management. Why are we building complex vector stores when we could just use markdown files with Git's built-in diff/blame/history? How it works: Memories stored as markdown files in a Git repo Each conversation = one commit git diff shows how understanding evolves over time BM25 for search (no embeddings needed) LLMs generate search queries from conversation context Example: Ask "how has my project evolved?" and it uses git diff to show actual…
2025 · github.com · its alternatives →
15
Model2vec – Lightning-fast Static Embeddings for RAG/Semantic Search▲28
We’ve recently open-sourced Model2vec, a method to distill sentence transformers into static embeddings that outperform all previous approaches by a large margin on MTEB. Our new models set a new state-of-the-art for static embeddings. Main features: - Our best model (potion-base-8M) has only 8M parameters, which is ~30mb on disk - Inference is ~500x faster than the distilled base model (bge-base), on a CPU - New models can be distilled in 30 seconds on a CPU without requiring a dataset - just a vocabulary - Numpy-only inference: The packaged can be install the package with minimal…
2024 · github.com · its alternatives →
16
We built an AI to review your pull requests▲53
Hey HN, We’re two developers (co-founders) with a team of 20 who got tired of spending hours reviewing PRs, so we built Infinitcode.ai, an AI-powered code reviewer that: - *Summarizes PRs in plain English*: No more deciphering 1,000-line diff jungles - *Catches more than bugs*: Security holes, performance pitfalls, code smells, even typos (yes, we’ll flag “vurnerabilities” and vulnerabilities) - *Zero onboarding*: Works instantly—no “let me learn your codebase for weeks” nonsense. Why we’re posting: We’re in alpha and need brutal honesty. Roast our tool, mock our UI, or tell us why AI will…
2025 · infinitcode.ai · its alternatives →
17
Embeddinghub: A vector database built for Machine Learning embeddings▲118
The Virtual Feature Store. Turn your existing data infrastructure into a feature store. - featureform/featureform
2021 · github.com · its alternatives →
18
Model2vec-Rs – Fast Static Text Embeddings in Rust▲60
Hey HN! We’ve just open-sourced model2vec-rs, a Rust crate for loading and running Model2Vec static embedding models with zero Python dependency. This allows you to embed text at (very) high throughput; for example, in a Rust-based microservice or CLI tool. This can be used for semantic search, retrieval, RAG, or any other text embedding usecase. Main Features: - Rust-native inference: Load any Model2Vec model from Hugging Face or your local path with StaticModel::from_pretrained(...). - Tiny footprint: The crate itself is only ~1.7 mb, with embedding models between 7 and 30 mb. Performance:…
2025 · github.com · its alternatives →
19
Visual inference exploration and experimentation playground▲46
Most inference UIs that I've come across pretty much just give us a chat-like interface to toy around with models in a single visual conversation thread. Given the fact that we are limited to seeing only one output at a time, it's kind of hard to compare outputs from different models, adjustments made to the prompting, and sampler settings. But even when keeping the generation parameters the same (e.g., to test for reliability in the output) and just going for multiple passes, there is no easy way to have a side-by-side comparison to keep track of the outputs from the multiple "rounds". I…
2024 · github.com · its alternatives →
20
Embedditor [::]▲86
Improved vector search with open-source AI embeddings editor
2023 · its alternatives →
21
Monostate AItraining▲59
Fine-tuning, RL, and inference in one CLI
Dec 2025 · github.com · its alternatives →
22
Dockerized GPU Deep Learning Solution (Code and Blog and TensorFlow Demo)▲80
An on-premises, bare-metal solution for deploying GPU-powered applications in containers - emergingstack/es-dev-stack
2016 · github.com · its alternatives →
23
Qdrant Cloud Inference▲22
Unify embeddings and vector search across modalities
2025 · qdrant.tech · its alternatives →
24
I wrote an autodiff in C++ and implemented LeNet with it▲36
2024 · gitlab.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →