AI · alternatives · 2026

24 alternatives to OpenBench

Benchmark local LLMs without living in the terminal.

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
Find the best local LLM for your hardware, ranked by benchmarks▲283
Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm
May 2026 · github.com · its alternatives →
2
LocalScore – Local LLM Benchmark▲124
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai · its alternatives →
3AT
A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)▲55
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
4
LLM Stats▲293
I was overspending on GPT-4o. It was really hard to compare different models I could switch to, so I built this LLM comparison tool. It shows leaderboards, pricing, and performance data across 100+ LLMs (including all major providers and open-source models). Key features: - Live pricing comparisons - Benchmark Scores (MMLU, HumanEval, GPQA, etc.) - Context length vs cost analysis - Speed/throughput tests across providers - Quality vs price visualizations - Open source (all data verifiable) Try it out: https://llmstats.com I'd like to know your opinion :) Tech stack: Next.js,…
Oct 2025 · llm-stats.com · its alternatives →
5
Llmspeed – feel how fast a local LLM runs before you buy the hardware▲329
Pick hardware and a model, and feel the real speed of a local LLM before you buy. Simulates tokens/sec and time-to-first-token from verified benchmarks for GPUs, Macs, and more.
5d ago · llmspeed.dev · its alternatives →
6UD
Unify – Dynamic LLM Benchmarks and SSO for Multi-Vendor Deployment▲91
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024 · its alternatives →
7
oqoqo▲325
Most benchmarks today exist in curated environments and do not translate well to the real world. We built Oqoqo to bridge this gap. Oqoqo makes it super simple to build realistic evals and custom benchmarks for tasks users actually care about. Oqoqo can: - Reliably measure how agent friendly your product surfaces are against Codex, Claude Code, OpenClaw, Hermes, Pi, Opencode, Cursor, GitHub Copilot - Regression test MCP, CLI, skills, SDK, and any agent facing interface (we are continuously using Oqoqo to dogfood and improve our own MCP/CLI) - Create and share custom benchmarks for how…
Aug 2026 · oqoqo.ai · its alternatives →
8
finetune LLMs via the Finetuning Hub▲80
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for…
2023 · github.com · its alternatives →
9
UBenchan▲55
Beautiful browser benchmarks
2022 · its alternatives →
10
BenchLLM by V7▲133
Test-driven development for LLMs
2023 · benchllm.com · its alternatives →
11
OpenLLMetry – OpenTelemetry-based observability for LLMs▲154
Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the…
2023 · github.com · its alternatives →
12
Ollama LLM Throughput Benchmark▲9
Measure & Maximize Ollama LLM Performance Across Hardware
2025 · llm.aidatatools.com · its alternatives →
13
You don't need to adopt new tools for LLM observability▲102
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com · its alternatives →
14
Deepmark AI▲97
LLM benchmarking tool for task-specific metrics on your data
2023 · github.com · its alternatives →
15
Bench – A generic latency benchmarking library▲37
A generic latency benchmarking library. Contribute to tylertreat/bench development by creating an account on GitHub.
2015 · github.com · its alternatives →
16
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
17
Inferbench▲2
Benchmarks local LLM engines on your hardware
Aug 2026 · github.com · its alternatives →
18
I built a tool to check if your computer can run LLMs locally▲8
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com · its alternatives →
19CI
Can I run this LLM? (locally)▲42
One of the most frequent questions one faces while running LLMs locally is: I have xx RAM and yy GPU, Can I run zz LLM model ? I have vibe coded a simple application to help you with just that. Update: A lot of great feedback for me to improve the app. Thank you all.
2025 · can-i-run-this-llm-blue.vercel.app · its alternatives →
20TR
Trunchbull, run real models against any benchmark in your browser▲6
Hi HN, Today I'm showcasing Trunchbull, a benchmarking platform designed for authoring benchmarks and running them against different models. We have direct support for benchmarks that use the harbor authoring system, custom tool authoring via the vercel ai sdk and configuration limits. We've also already imported terminalbench 2.0, as a sort of proof of concept that our harbor task orchestrator works, although you currently need a paid account as we are provisioning sandbox environments. I've made several popular benchmarks publicly available for testing. You dont need an account or your…
Aug 2026 · trunchbull.dev · its alternatives →
21
Open Source LLM Performance Tracker▲28
An open source Next app template to monitor your AI apps
2025 · github.com · its alternatives →
22
Benchi – A benchmarking tool written in Go▲58
Benchi is a CLI tool for running benchmarks and collecting metrics. It's using Docker Compose to orchestrate the infrastructure and tools being benchmarked, making it repeatable and runnable on different machines. It allows you to run the same benchmark for different tools and compare the collected results. The repository contains a simple example. For a more elaborate example see how we use Benchi to compare data pipelines running on Conduit and Kafka Connect, two data streaming tools (still work in progress): https://github.com/ConduitIO/streaming-benchmarks
2025 · github.com · its alternatives →
23
LLM Tree Navigation Benchmark▲6
Measures the ability of various LLMs to navigate a fictional codebase via iterative directory tree expansion and observation. Each model's baseline ability is compared against combinations of various prompt engineering mods to quantify exactly how much they help or hinder the LLM. Interesting findings here: https://github.com/aiwebb/treenav-bench#interesting-findings
2024 · github.com · its alternatives →
24
OpenLIT – Open-Source LLM Observability with OpenTelemetry▲62
Hey HN, we're super excited to share something we've been working on: OpenLIT. After an engaging preview that some of you might recall, we are now proudly announcing our first stable release! *What's OpenLIT?* Simply put, OpenLIT is an open-source tool designed to make monitoring your Large Language Model (LLM) applications straightforward. It’s built on OpenTelemetry, aiming to reduce the complexities that come with observing the behavior and usage of your LLM stack. *Beyond Basic Text Generation:* OpenLIT isn’t restricted to just text and chatbot outputs. It now includes automatic…
2024 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →