Dev tools · alternatives · 2026

24 alternatives to Optimizing LiteLLM with Rust – When Expectations Meet Reality

I've been working on Fast LiteLLM - a Rust acceleration layer for the popular LiteLLM library - and I had some interesting learnings that might resonate with other developers trying to squeeze performance out of existing systems. My assumption was that LiteLLM, being a Python library, would have plenty of low-hanging fruit for optimization. I set out to create a Rust layer using PyO3 to accelerate the performance-critical parts: token counting, routing, rate limiting, and connection pooling. The Approach - Built Rust implementations for token counting using tiktoken-rs - Added lock-free data…

Fast LiteLLM is a Rust acceleration layer that speeds up the Python LiteLLM library for developers working with large language models. It optimizes performance-critical functions including token counting, routing, rate… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
Glidesort, a new stable sort in Rust up to ~4x faster for random data▲389
Hi all, I've talked about glidesort a few times on HN already, but it's finally ready for release. If you have any questions, feel free to ask. An academic paper on glidesort that goes into a lot more detail than the readme is upcoming, but is not ready yet. I will be giving a talk on glidesort tomorrow at FOSDEM 2023 in the Rust Devroom at 16:10, you can seek me out there as well. In other news, I am leaving academia soon, so if you have interesting (Rust) jobs the coming months feel free to approach me.
2023 · github.com · its alternatives →
2
Sub-millisecond VM sandboxes using CoW memory forking▲311
I wanted to see how fast an isolated code sandbox could start if I never had to boot a fresh VM. So instead of launching a new microVM per execution, I boot Firecracker once with Python and numpy already loaded, then snapshot the full VM state. Every execution after that creates a new KVM VM backed by a `MAP_PRIVATE` mapping of the snapshot memory, so Linux gives me copy-on-write pages automatically. That means each sandbox starts from an already-running Python process inside a real VM, runs the code, and exits. These are real KVM VMs, not containers: separate guest kernel, separate guest…
Mar 2026 · github.com · its alternatives →
3
Tantivy▲68
A full-text, horse-speed search engine library in Rust
2022 · github.com · its alternatives →
4
A Fast, Malloc-Free C++14 Json Parser and Encoder▲98
Meta programming utilities for C++14. Merged in matt-42/lithium - matt-42/iod
2014 · github.com · its alternatives →
5PA
Python at the Speed of Rust▲68
I’m sure many of you are familiar, but there’s a treacherous gap between finding (or building) a model that works in PyTorch, and getting that deployed into your application, especially in consumer-facing applications. I’ve been very interested in solving this problem with a great developer experience. Over time, I gradually realized that the highest-impact thing to have was a way to go from existing Python code to a self-contained native binary—in other words, a Python compiler. I was already pretty familiar with a successful attempt: when Apple introduced armv8 on the iPhone 5s, they…
2025 · blog.fxn.ai · its alternatives →
6
A fast HNSW implementation in Rust▲111
HNSW implementation in Rust. Reference: https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf - swapneel/hnsw-rust
2024 · github.com · its alternatives →
7
I wrote a tool in Rust for tracking all allocations in a Linux process▲113
allocscope - a memory tracking tool. Contribute to matt-kimball/allocscope development by creating an account on GitHub.
2023 · github.com · its alternatives →
8
Faking SIMD to Search and Sort Strings 5x Faster▲29
I want to share a really dumb, but very practical project I have packaged this summer, to perform operations on strings much faster. I was using Python to work with a multi-terabyte newline-delimited file. Reading, splitting, and shuffling it was a nightmare. So, I wrapped a trivial hardware-friendly heuristic I've been using for the last few years into a CPython library. The part I enjoyed the most is implementing SIMD behavior without SIMD instructions... Using 64-bit words to work at 8-bit granularity. Unlike conventional SIMD, the code would remain the same for ~~almost~~ any hardware.…
2023 · ashvardanian.com · its alternatives →
9
I built a Rust crate for running unsafe code safely▲113
A Rust crate for containing memory leaks and fragmentation - brannondorsey/mem-isolate
2025 · github.com · its alternatives →
10AF
Apache Fory Rust – 10-20x faster serialization than JSON/Protobuf▲67
Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf. Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs. Unique features that other fast serializers don't have: - Cross-language without IDL files (Rust ↔ Python/Java/Go) - Trait object serialization (Box) - Automatic circular reference handling - Schema evolution without coordination Happy to discuss design trade-offs. Benchmarks:…
Oct 2025 · fory.apache.org · its alternatives →
11
A SIMD-accelerated noise library in Rust▲73
SIMD noise library for Rust. Contribute to verpeteren/rust-simd-noise development by creating an account on GitHub.
2018 · github.com · its alternatives →
12
Second State VM▲78
High performance secure & portable Rust functions in Node.js
2020 · secondstate.io · its alternatives →
13
Rust macro utility for batching expensive async operations▲28
rust macro util for batching expensive operations. Contribute to hackermondev/batched development by creating an account on GitHub.
2025 · github.com · its alternatives →
14SA
Spart – A Rust library for fast spatial search with Python bindings▲31
Hi everyone, I've made an open-source library for fast spatial search in Rust. It's called Spart, and it currently provides the following features: - Five tree implementations: Quadtree, Octree, Kd-tree, R-tree, and R*-tree - Python bindings (`pyspart` on PyPI) - Fast k-nearest neighbor (kNN) and radius search - Bulk data loading for efficient tree construction Project's GitHub repo: https://github.com/habedi/spart
2025 · its alternatives →
15
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
16
Python JSON library in Rust, faster than ujson▲40
Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy - ijl/orjson
2018 · github.com · its alternatives →
17
I embedded 685M public texts in 32 minutes (on 8x A100, Rust, TensorRT)▲7
Quick note on how it works and how I've done my batch embedding engine IgniteMS. The whole thing runs as one process using Rust, reading input, tokenizing, packing batches, keeping the queue full. TensorRT handles inference. Python is only as a wrapper. I built it this way because when you use more than couple of GPUs, the GPUs stop being the problem. CPU cannot feed them fast enough. One A100 can go through batches faster than Python can tokenize and feed, so the GPU just sits there idle waiting for work. Most of my time went into optimizing this. At 8 GPUs that was basically the entire…
Jun 2026 · github.com · its alternatives →
18
Pyrtls, rustls-based modern TLS for Python▲22
rustls-based modern TLS for Python. Contribute to djc/pyrtls development by creating an account on GitHub.
2024 · github.com · its alternatives →
19T5
TensorBase: 5x~10000x Faster Drop-In/Accelerator for ClickHouse in Rust▲19
2021 · tensorbase.io · its alternatives →
20
I made an open-source Rust program for memory-efficient genomics▲17
My cofounder and I run a startup in oncology, where we handle cancer genomics data. It occurred to me that, thanks to a recent complexity theory result, there's a clever way to run bioinformatics algorithms using far less RAM. I built this Rust engine for running whole-genome workloads in under 100MB of RAM. Runtime is a little longer as a result - O(TlogT) instead of O(T). But it should enable whole-genome analytics on consumer-grade hardware.
Nov 2025 · github.com · its alternatives →
21
Achilles – Automatic profiling and C++ optimization of Python via LLMs▲7
Hey HN! We built Achilles, a tool that automatically accelerates your Python code. It identifies performance bottlenecks, rewrites those functions in optimized C++, and seamlessly patches them into your running program—without you changing a single line of code. In CPU-intensive, loop-heavy tasks, we've observed performance improvements of 100-1000x. Achilles can be installed via pip and works with just a single command. We'd appreciate your feedback, and feel free to give us a star if you find it interesting!
2025 · github.com · its alternatives →
22HF
Hurry – Fast Rust build caching▲11
Hey HN, we’re Eliza and Xin. We’re working on Hurry, an open source drop-in tool that adds distributed build caching to Cargo with (almost) zero configuration. Wherever you run cargo build, you can run hurry cargo build instead, and expect around 2-5x (our best benchmark is 22x) faster builds. We built this because we were dissatisfied with the current build caching options available for Rust. Buck and Bazel require learning a new tool. GitHub Actions and swatinem are too coarse-grained (either the whole cache hits, or none of it does) and finicky to debug, and cargo-chef and Docker layer…
Dec 2025 · hurry.build · its alternatives →
23
Fast subdomain enumerator written in Rust▲23
Hello! I'm learning Rust and also information security, so what better way to learn than to put a tool together practicing both skills? I've tried to use a couple of different tools for subdomain enumeration (part of the Discovery-party when it comes to penetration testing), but none of them were very fast, even if the DNS resolver I pointed them towards is, so I wrote my own tool that leverages concurrency to check a lot of subdomains, fast! It's written in Rust, main parts being using Tokio and async_channels, and I think it works good enough for a first release. I wanted to get recursive…
2022 · github.com · its alternatives →
24SS
SSVM – Speed up your Node.js apps with Rust and WebAssembly▲5
Raw performance (the lack of) is always a contentious point for JavaScript, especially in our post-Moore era of computing. Node.js gets around this issue through extensive of use of native libraries and modules written in C, including the V8 JS engine itself. But as application developers, how do YOU create high performance functions? Write in C and run as a NAPI module? Write in Rust and run as a V8 wasm? Well, there is another (better and faster) way now! The SSVM is a fully featured WebAssembly VM that is optimized for server-side applications.…
2020 · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →