Bhumi–OSS Python Library w Rust Underhead for 2.5x Faster LLM Inference
Read the full blogpost at https://rach.codes/blog/Introducing-Bhumi (click on reader to see the technical breakdown!) AI inference should be fast, but in practice it’s painfully slow. Inference bottlenecks slow down LLM-powered chatbots and AI workflows everywhere. I built Bhumi to fix that. Bhumi is a Python library designed for developers, yet its performance-critical core is implemented in Rust (via PyO3) for near-native speed. This hybrid approach delivers up to 2.5x faster response times across providers like OpenAI, Anthropic, and Gemini—without changing the…
Bhumi–OSS Python Library w Rust Underhead for 2.5x Faster LLM Inference launched on February 24, 2025 with 8 votes and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
Bhumi is the blazing-fast AI inference client for Python. Built with Rust for unmatched performance, supporting OpenAI, Anthropic, Gemini, Groq, and more. Get 3x faster inference with 60% less memory usage.
Built with Rust for performance. Unified interface for OpenAI, Anthropic, Gemini, Groq , and more. 1 from bhumi . base_client import BaseLLMClient , LLMConfig 2 import asyncio 3 4 async def main ( ) : 5 config = LLMConfig ( 6 api_key = "YOUR_OPENAI_API_KEY" , 7 model = "openai/gpt-4o" , 8 max_tokens = 1000 9 ) 10 11 client = BaseLLMClient ( config ) 12 13 response = await client . completion ( [ 14 { "role" : "user" , "content" : "Explain quantum computing" } 15 ] ) 16 17 print ( response [ 'text' ] ) 18 19 asyncio . run ( main ( ) ) Quick Start Get started with your first AI call in seconds 1 from bhumi . base_client import BaseLLMClient , LLMConfig 2 import asyncio 3 4 async def main ( )…from bhumi.trilok.ai
In the maker’s words, at launch
Read the full blogpost at https://rach.codes/blog/Introducing-Bhumi (click on reader to see the technical breakdown!) AI inference should be fast, but in practice it’s painfully slow. Inference bottlenecks slow down LLM-powered chatbots and AI workflows everywhere. I built Bhumi to fix that. Bhumi…
Does a similar job
all alternatives →
We made our own inference engine for Apple Silicon2025 · github.com · ▲186We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
Faster FastAPI with simdjson and io_uring on Linux 5.192023 · github.com · ▲290A few months ago, I benchmarked FastAPI on an i9 MacBook Pro. I couldn't believe my eyes. A primary REST endpoint to `sum` two integers took 6 milliseconds to evaluate. It is okay if you are targeting a server in another city, but it should be less when your client and server apps are running on the same machine. FastAPI would have bottleneck-ed the inference of our lightweight UForm neural networks recently trending on HN under the title "Beating OpenAI CLIP with 100x less data and compute". (Thank you all for the kind words!) So I wrote another library. It has been a while since I have…
Timber – Ollama for classical ML models, 336x faster than PythonMar 2026 · github.com · ▲207Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference. - kossisoroyce/timber
General ComputeMay 2026 · generalcompute.com · ▲315AI models that run on an inference cloud optimized for speed
TensorZero2025 · github.com · ▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, February 2025
the whole month →
Screen Studio 3.0▲1,832Beautiful screen recordings with instant shareable links
Growth · 2025 · screen.studio
- IG
I was at FB/Meta from late 2013 to early 2023, mostly working in the compiler/runtime spaces. I got hit in the spring 2023 layoff wave. I immediately started making games in my newfound free time (a lifelong interest, and I even worked in AA(A?) back ca. ~2000), and in October 2023 I stumbled upon the idea of a roguelike pachinko/plinko game inspired by Luck Be A Landlord. Things snowballed quickly, I started talking to publishers, then worked like crazy through all of 2024, almost the hardest I've ever worked in my career, and launched the game in December 2024. It's sold…
Work · 2025
i wanted to change the habit of reaching for my phone in the morning and doomscrolling away an hour so i built an app to help me. now i have to literally touch grass before accessing my most distracting apps the app is built in swiftui, uses the screen time apis provided by apple and google vision to recognise grass or not i'd love to get your thoughts on the concept.
Life & fun · 2025 · touchgrass.now