AI · alternatives · 2026
24 alternatives to Groq®
Hyperfast LLM running on custom built GPUs
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Groq® launched in 2024; newer entries below may have overtaken it.
- 1
General Compute▲315AI models that run on an inference cloud optimized for speed
May 2026 · generalcompute.com · its alternatives →
- 2
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
- 3
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
- 4GA
2021 · inferrd.com · its alternatives →
- 5
The world’s most powerful chip’ for AI
2024 · nvidia.com · its alternatives →
- 6IB
We had no guide or course that teaches chip design at our university. We had taken a digital logic course, but were disappointed with the fact that the most complex project we did was building a full adder in Quartus using logic blocks, not even in RTL!5 Therefore, we decided to challenge ourselves to dive deep into machine learning (ML) hardware and learn as much as we could on our own. We wanted to prove that basic math (like y = mx + b) and basic logic circuits are enough to help anyone understand how modern AI hardware works. Our goal was to design our own version of the LPU from scratch…
Aug 2026 · lpulite.com · its alternatives →
- 7
Fast multimodal-native inference at scale
Dec 2025 · gmicloud.ai · its alternatives →
- 8
- 9
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com · its alternatives →
- 10
High performance storage engine for efficient LLM inference and GPU Training.
Sep 2026 · theopenlake.com · its alternatives →
- 11
- 12
- 13
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 14
SelfHostLLM▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
- 15
Powers faster, efficient reasoning for long-running agents
Jun 2026 · developer.nvidia.com · its alternatives →
- 16
Fast LLMs for low-latency and high-performance workflows
Jun 2026 · jetbrains.com · its alternatives →
- 17
Serve fine-tuned SLMs 4x faster for 50% less cost.
2024 · rubrik.com · its alternatives →
- 18
- 19
- 20
Deploying vision models is time consuming and tedious. Setting up dependencies. Fixing conflicts. Configuring TRT acceleration. Flashing (and re-flashing) NVIDIA Jetsons. A streamlined, developer-friendly solution for inference is needed. We, the Roboflow team, have been hard at work open sourcing Inference, an open source vision deployment solution. Our solution is designed with developers in mind, offering a HTTP-based interface. Run models on your hardware without having to write architecture-specific inference code. Here's a demo showing how to go from a model to GPU inference on a video…
2023 · github.com · its alternatives →
- 21
- 22
High-Performance GPU Cuckoo Filter. Contribute to tdortman/Cuckoo-GPU development by creating an account on GitHub.
Jan 2026 · github.com · its alternatives →
- 23
CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included. - swellweb/reame
Jul 2026 · github.com · its alternatives →
- 24
Inference hosting for AI teams who ship fast and scale faster.
2022 · banana.dev · its alternatives →
Also compare
- General Compute alternatives
- Tiny-vLLM – high performance LLM inference engine in C++ and CUDA alternatives
- Speeding up LLM inference 2x times (possibly) alternatives
- GPU-Accelerated Inference Hosting alternatives
- Nvidia Blackwell B200 GPU alternatives
- I built a lite LPU that can do inference on Karpathy's MicroGPT alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →