Alternatives
Products that do what Trunchbull, run real models against any benchmark in your browser does
Hi HN, Today I'm showcasing Trunchbull, a benchmarking platform designed for authoring benchmarks and running them against different models. We have direct support for benchmarks that use the harbor authoring system, custom tool authoring via the vercel ai sdk and configuration limits. We've also already imported terminalbench 2.0, as a sort of proof of concept that our harbor task orchestrator works, although you currently need a paid account as we are provisioning sandbox environments. I've made several popular benchmarks publicly available for testing. You dont need an account or your…
- 1OA
Scored 65.2% vs google's official 47.8%, and the existing top closed source model Junie CLI's 64.3%. Since there are a lot of reports of deliberate cheating on TerminalBench 2.0 lately (https://debugml.github.io/cheating-agents/), I would like to also clarify a few things 1. Absolutely no {agents/skills}.md files were inserted at any point. No cheating mechanisms whatsoever 2. The cli agent was run in leaderboard compliant way (no modification of resources or timeouts) 3. The full terminal bench run was done using the fully open source version of the agent, no…
Apr 2026 · github.com
- 2LL
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai
- 3TB
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent…
2025 · github.com
- 4
- 5UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024
- 6AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 7
- 8
- 9HG
Tabs, splits, and tmux work fine until you have several projects open with logs, tests, and long-running shells. I kept rebuilding context instead of resuming work. Horizon puts shells on an infinite canvas. You can arrange them into workspaces and reopen later with layout, scrollback, and history intact. Built in 3 days with Claude/Codex, dogfooding the workflow as I went. Feedback and contributions welcome.
Mar 2026 · github.com
- 10BR
I built BenchFlow, an open-source framework that lets you integrate and evaluate AI tasks using Docker-based benchmarks. You can try it out right now by cloning the repo and running a benchmark in minutes. As an AI researcher, I was frustrated with how much time my team spent setting up benchmark environments rather than actually improving our models. We'd spend weeks configuring environments, only to find inconsistencies when comparing results with other teams. BenchFlow started as an internal tool to standardize our evaluation process, and we decided to open-source it after seeing how much…
2025 · github.com
- 11IM
I built BuzzBench because I was frustrated with how complex performance testing tools have become. And I was ending up writing my own scripts to test endpoints and manually check out resource usage at the time of testing. Checkout demo: https://www.youtube.com/watch?v=yAnbZMoQvmQ
2025 · buzzbench.io
- 12TW
I want to share a new dataset of 331 reward-hackable environments. These are real environments used in Terminal Bench and adjacent benchmarks. I first got interested in this because, as a reviewer of Terminal Bench, I noticed a lot of our tasks were hackable. I also noticed that many contributors to the benchmark do so because it provides credibility when selling environments to labs. Hence, TBench tasks are, in my opinion, held to a higher quality standard than those being used today for RL. No one is spending hours manually reviewing the $1B in tasks being purchased by major labs. As far…
Apr 2026 · github.com
- 13
I'm Fenil, co-founder/CEO of OpenFunnel (YC F24), building this with my co-founder/CTO Aditya. We're launching OpenBenchmarks (https://openbenchmarks.com), open-source, reproducible benchmarks for SaaS APIs, starting with the category we know best: GTM APIs. ## Why we built this More and more B2B software evaluation will/already runs through reasoning models inside agentic workflows rather than through people. And buyers increasingly pick vendors that are API-first and ship MCPs, so they can wire them into internal workflows. Strong reasoning models are skeptical of…
Jul 2026 · openbenchmarks.com
- 14AR
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
2024 · explorer.invariantlabs.ai
- 15TI
I'm an "ideas person" who messes around with AI on a low budget. I got tired of watching my tokens vanish and context windows filling up while agents fumbled around trying to find the right thing. Agents don't flail like they used to with shell tools, but there are still weak/blind spots and back-and-forth episodes — especially when using tools in combination/sequence. So I built "tilth" today. Or rather, AI built it — every line is Opus 4.6. I spent a lot of my precious tokens getting it to "not shit" (at least several of the different vendors' AI overlords assure me it's not…
Feb 2026 · github.com
- 16DM
Hey Hacker News, We're the maintainers of docker/model-runner and wanted to share some major updates we're excited about. Link: https://github.com/docker/model-runner We are rebooting the community: https://www.docker.com/blog/rebooting-model-runner-community... At its core, model-runner is a simple, backend-agnostic tool for downloading and running local large language models. Think of it as a consistent interface to interact with different model backends. One of our main backends is llama.cpp, and we make it a point to contribute any…
Oct 2025 · github.com
- 17WB
Over the past few months, as we scaled our internal AI Agents, we hit a dead end: Running LLM-generated arbitrary code in Docker is basically running naked on security due to container escape risks. But using full traditional VMs takes minutes to boot and eats too much memory to support high-density concurrency. We loved the developer experience of SaaS sandboxes on the market, but they are closed-source, expensive, and have too high a barrier to entry for self-hosting. So, our team decided to build our own. After months of grinding, using RustVMM and KVM, we built a blazing-fast,…
Apr 2026 · github.com
- 18
- 19AD
I'd like to share a project I've been working on for the past few months. It's a distributed workflow engine written entirely in Go. Some highlights: * Tasks are executed in a Docker container * Can run stand-alone or distributed * Highly extensible * Able to enforce limits (CPU/RAM) per task * Web UI Would love the get your feedback on it, and find out if this could be useful.
2023 · github.com
- 20IM
Hey HN! I made a completely open sourced alternative to Weights and Biases with (insert cringe) blazingly fast performance (yes we use rust and clickhouse) Weights and Biases is super unperformant, their logger blocks user code... logging should not be blocking, yet they got away with it. We do the right thing by being non blocking. Would love any thoughts / feedbacks / roasts etc
2025 · github.com
- 21CC
If you had to build a context window manager in 24h, would you stick to the existing model or come up with something better? Here's what I did: 1. Built a proxy that intercepts Codex's calls to OpenAI and rewrites them on the fly. 2. Replayed 3,807 rounds of SWE-bench Verified traces through it: avg prompt 44k → 6k tokens (-87%). 3. Posted it to HN to get the next reduction applied to my confidence interval — starting with the inevitable "How about accuracy?" npx -y pando-proxy · github.com/human-software-us/pando-proxy
Apr 2026 · npmjs.com
- 22
- 23CB
I built a small benchmark to test CLI coding agents on blind bug detection. A challenger agent injects bugs and writes ground truth (`bugs.json`). A different reviewer agent audits the repo without seeing ground truth, and an LLM matcher scores bug-to-finding assignments. Current run: 50 repos, 150 challenges, 450 reviews, 2,603 injected bugs. Weighted detection: Claude 58.05%, Codex 37.84%, Gemini 27.81%. LLM-judge benchmarks are easy to get wrong, so I’d really appreciate critical feedback on benchmark fairness, scoring/matching methodology, and obvious failure modes I’m missing. Full…
Feb 2026 · github.com
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →