Dev tools · alternatives · 2026

24 alternatives to gline-rs – an inference engine for GLiNER models, in Rust

Hi everyone, wanted to share about gline-rs, an inference engine for GLiNER models written in Rust. This family of lightweight language models proved to be efficient at zero-shot Named Entity Recognition (NER) and other tasks such as Relation Extraction, while consuming less resources than large generative models (LLMs). This implementation has been written from the ground up in Rust, and supports both span- and token-oriented variants (for inference only). The goal is to provide a production-grade and user-friendly API in a modern and safe programming language, including a clean and…

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
We made our own inference engine for Apple Silicon▲186
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com · its alternatives →
2
Model2vec-Rs – Fast Static Text Embeddings in Rust▲60
Hey HN! We’ve just open-sourced model2vec-rs, a Rust crate for loading and running Model2Vec static embedding models with zero Python dependency. This allows you to embed text at (very) high throughput; for example, in a Rust-based microservice or CLI tool. This can be used for semantic search, retrieval, RAG, or any other text embedding usecase. Main Features: - Rust-native inference: Load any Model2Vec model from Hugging Face or your local path with StaticModel::from_pretrained(...). - Tiny footprint: The crate itself is only ~1.7 mb, with embedding models between 7 and 30 mb. Performance:…
2025 · github.com · its alternatives →
3
An educational Local Qwen3 LLM Inference project written in Rust▲15
An educational Rust project for exporting and running inference on Qwen3 LLM family - reinterpretcat/qwen3-rs
2025 · github.com · its alternatives →
4
Timber – Ollama for classical ML models, 336x faster than Python▲207
Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference. - kossisoroyce/timber
Mar 2026 · github.com · its alternatives →
5
cuTile Rust: Safe, data-race-free GPU kernels in Rust▲106
cuTile Rust provides a safe, tile-based kernel programming DSL for the Rust programming language. It features a safe host-side API for passing tensors to asynchronously executed kernel functions. - NVlabs/cutile-rs
Jun 2026 · github.com · its alternatives →
6SA
Spice.ai OSS 1.0 – data query and AI-inference engine built in Rust▲26
2025 · spiceai.org · its alternatives →
7
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
8
Rust framework for advanced file recognition and identification▲46
Alternative to magic.h and infer. Zero dependencies. Fully extensible. Works in no_std, async, and embedded contexts.
2025 · crates.io · its alternatives →
9
Kalosm an embeddable framework for pre-trained models in Rust▲66
Hi everyone, I'm happy to announce the release of Kalosm! [Kalosm](https://floneum.com/kalosm/) is a framework for embedded AI in rust. ## What is Kalosm? Kalosm provides a simple interface for pre-trained language, audio, and image models models. To make it easy to use with these models in your application, Kalosm includes a set of integrations other systems like your database or documents. ```rust use kalosm::language::*; #[tokio::main] async fn main() { let mut model = Llama::new_chat(); let mut chat = Chat::builder(&mut model) .with_system_prompt("The assistant will…
2024 · floneum.com · its alternatives →
10
a Rust library for creating hierarchical state machines▲38
I've come to prefer modeling state machines with nested states, but implementing them by hand requires a large amount of error-prone boilerplate that obscures the important logic of the state machine. I found that the existing offerings in the Rust ecosystem - while inspiring - lacked a number of features that I was looking for, so I spent the last few months building moku. Feedback is welcome!
2025 · docs.rs · its alternatives →
11LA
Lingo – A linguistic database in Rust with nanosecond-level performance▲42
Hi HN, I made Lingo - the SQLite of semantic search. I'm a self-taught developer and researcher who left school at 16, and I've spent some time exploring a first-principles approach to system design for various frontier problems. In this case it's AI that challenges the 'bigger is better' transformer paradigm. Lingo is the first piece of that research, a high-performance linguistic database designed to run on-device. The full technical overview and manifesto is here: https://medium.com/@robm.antunes/bcd1e9752af6 The paper has been archived on Zenodo with a DOI:…
2025 · its alternatives →
12
FemtoGPT – Pure Rust implementation of a GPT language model▲19
Pure Rust implementation of a minimal Generative Pretrained Transformer - keyvank/femtoGPT
2023 · github.com · its alternatives →
13
val – An arbitrary precision calculator language▲34
Wrote this to learn more about the `chumsky` parser combinator library, rustyline, and the `ariadne` error reporting crate. Such a nice DX combo for writing new languages. Still a work in progress, but I thought I'd share :)
2025 · github.com · its alternatives →
14
Machine learning (entirely) in Rust▲20
Machine Learning library for Rust. Contribute to AtheMathmo/rusty-machine development by creating an account on GitHub.
2016 · github.com · its alternatives →
15
Spice.ai Open Source 1.0-stable▲5
A portable data query and LLM-inference engine built in Rust
2025 · github.com · its alternatives →
16
Spanda – Sub-microsecond LLM epistemic uncertainty in Rust▲13
Zero-cost epistemic uncertainty quantification & hallucination detection for LLMs (90,000x faster than Semantic Entropy) - Adarshent/Spnda
Sep 2026 · github.com · its alternatives →
17
A book introducing Geospatial and Rust▲11
My friend and I have been collaborating on this lighthearted introduction to doing geospatial computation in rust. Don't know Rust? Don't know geospatial? I'd be interested to know what you think.
2022 · book.georust.org · its alternatives →
18
KTransformers–236B Model and 1M Context LLM Inference on Local Machines▲20
Hey Hacker News! We are excited to share our open-source project, KTransformers, a flexible framework designed for cutting-edge LLM inference optimizations! Leveraging state-of-the-art kernels from llamafile and marlin, KTransformers seamlessly enhances the performance of HuggingFace Transformers, making it possible to operate large 236B MoE models or extremely long 1M context locally with promising speed. KTransformers is a Python-centric framework designed with extensibility at its core. By implementing and injecting an optimized module with a single line of code, users gain access to a…
2024 · github.com · its alternatives →
19
React-like Declarative DSL for building synthetic LLM datasets▲10
We were frustrated with dataset generation DX: ad‑hoc scripts and JSON templates break as flows branch, tool‑calls drift, and reproducibility is hard. So we built Torque: a schema‑first, declarative, fully typesafe DSL to generate conversational datasets. What it is: Declarative DSL - Compose conversations like React components Fully Typesafe - Zod schemas with complete type inference Provider Agnostic - Generate with any AI SDK provider (OpenAI, Anthropic, DeepSeek, vLLM, LLaMA.cpp etc.) AI-Powered Content - Generate realistic varied datasets automatically without complicated scripts Faker…
Nov 2025 · github.com · its alternatives →
20
Shimmy – 5MB privacy-first, local alternative to Ollama (680MB)▲14
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary. - Michael-A-Kuykendall/shimmy
2025 · github.com · its alternatives →
21
RunNburn – Run a 295B Moe from a 98GB GGUF on a 64GB RAM Desktop▲11
runNburn is an Apache-2.0 Rust inference engine for quantized GGUF models that are too big for your fast memory. The core idea: weights stay file-backed (mmap), host residency stays under an explicit byte budget (--ram-budget), and GPU caches are sized from detected free/total VRAM — never from device-name presets. There is no conversion step, no sidecar cache files, no silent requantization. The GGUF on disk is the single source of truth. The result that made me want to post this: Tencent's Hy3 (295B total / 21B active sparse MoE, a single 97.8 GiB Q2_K GGUF) runs on my desktop…
Jul 2026 · github.com · its alternatives →
22
LLGTRT: TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output▲6
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com · its alternatives →
23
Rusdantic▲5
A unified, high-performance data validation and serialization framework for Rust, inspired by Pydantic's ergonomics and powered by Serde.
Mar 2026 · github.com · its alternatives →
24
I Built a Groth16 ZK-Proof Verifier from Scratch Using Rust▲6
I've always been fascinated by cryptographic principles, so I decided to dive deep into zero-knowledge proofs and implement a Groth16 proof verifier entirely in Rust. This project validates cryptographic proofs and demonstrates how succinct, verifiable computations can be achieved. Using Rust's powerful libraries like arkworks, I implemented pairing checks, public input commitments, and proof validation logic while ensuring the code remains modular and efficient. I handled the complexities of affine group operations, scalar multiplications, and optimized public input verification to make…
2024 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →