Alternatives
Products that do what Benchi – A benchmarking tool written in Go does
Benchi is a CLI tool for running benchmarks and collecting metrics. It's using Docker Compose to orchestrate the infrastructure and tools being benchmarked, making it repeatable and runnable on different machines. It allows you to run the same benchmark for different tools and compare the collected results. The repository contains a simple example. For a more elaborate example see how we use Benchi to compare data pipelines running on Conduit and Kafka Connect, two data streaming tools (still work in progress): https://github.com/ConduitIO/streaming-benchmarks
- 1BR
I built BenchFlow, an open-source framework that lets you integrate and evaluate AI tasks using Docker-based benchmarks. You can try it out right now by cloning the repo and running a benchmark in minutes. As an AI researcher, I was frustrated with how much time my team spent setting up benchmark environments rather than actually improving our models. We'd spend weeks configuring environments, only to find inconsistencies when comparing results with other teams. BenchFlow started as an internal tool to standardize our evaluation process, and we decided to open-source it after seeing how much…
2025 · github.com
- 2DB
I built a CLI to benchmark DNS resolvers after discovering DNS was adding 300ms to my API requests. v0.3.0 just released with new features: compare: Test single domain across all resolvers top: Rank resolvers by latency/reliability/balanced monitor: Continuous tracking with threshold alerts 1,400+ downloads in first week. Quick start: pip install dns-benchmark-tool dns-benchmark compare --domain google.com CLI stays free forever. Hosted version (multi-region, historical tracking, alerts) coming Q1 2026. GitHub: https://github.com/frankovo/dns-benchmark-tool…
Nov 2025 · github.com
- 3LL
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai
- 4
- 5IB
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com
- 6
- 7
- 8AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 9
- 10
- 11
- 12
- 13MB
I always had a sweet tooth for how easy it is to use google/benchmark, but when working with js, current libraries didn't feel right and some were not even accurate enough, so I decided to create my own library to make JavaScript benchmarking tooling better. With more free time, I finally implemented all features I wished for in 1.0.0 and made a lightweight C++ single-header version for moments when google/benchmark is too much. Hope this library helps you as much as it does me.
2024 · github.com
- 14ST
I wanted to make my own evaluation of what kind of performance I could expect from SQLite on a server and investigate the experimental `BEGIN CONCURRENT` branch vs the inbuilt `DEFERRED` and `IMMEDIATE` behaviors. Explanatory blog post: https://reorchestrate.com/posts/sqlite-transactions/
2024 · github.com
- 15
- 16UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024
- 17CB
Hey HN, we're excited to share Cua-Bench ( https://github.com/trycua/cua ), an open-source framework for evaluating and training computer-use agents across different environments. Computer-use agents show massive performance variance across different UIs—an agent with 90% success on Windows 11 might drop to 9% on Windows XP for the same task. The problem is OS themes, browser versions, and UI variations that existing benchmarks don't capture. The existing benchmarks (OSWorld, Windows Agent Arena, AndroidWorld) were great but operated in silos—different harnesses,…
Jan 2026 · github.com
- 18WM
Hey, I'm Martyn and I recently joined Zuplo. OpenAPI is a huge part of what we do, but getting a document up to scratch out of the gate, especially if you're not super well versed in the actual specification and what you should use and when. So we built a suite of tools to help with this called Rate My OpenAPI. It will score your OpenAPI document out of 100, as well as giving you individual scores in 4 key areas; completeness, SDK generation, security and documentation. Along with the score, you also get details of what the issues, or errors are, as well as guidance on what they actually…
2024 · github.com
- 19
- 20MA
2024 · benchjs.com
- 21BI
2014 · github.com
- 22
- 23
- 24SP
2017 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →