Alternatives
Products that do what Benchmark AI on your actual code (GPT-5, Claude, Grok, Gemini, o3) does
I built CodeLens.AI - a tool that compares how 6 top LLMs (GPT-5, Claude Opus 4.1, Claude Sonnet 4.5, Grok 4, Gemini 2.5 Pro, o3) handle your actual code tasks. How it works: - Upload code + describe task (refactoring, security review, architecture, etc.) - All 6 models run in parallel (~2-5 min) - See side-by-side comparison with AI judge scores - Community votes on winners (blind voting) - Each evaluation gets reflected in the overall AI model leaderboard, showing us best ones Why I built this: Existing benchmarks (HumanEval, SWE-Bench) don't reflect real-world developer tasks. I wanted to…
- 1IR
2023 · sagittarius.greg.technology
- 2BV
Vision models have been gaining popularity as a replacement for traditional OCR. Especially with Gemini 2.0 becoming cost competitive with the cloud platforms. We've been continuously evaluating different models since we released the Zerox package last year (https://github.com/getomni-ai/zerox). And we wanted to put some numbers behind it. So we’re open sourcing our internal OCR benchmark + evaluation datasets. Full writeup + data explorer here: https://getomni.ai/ocr-benchmark Github: https://github.com/getomni-ai/benchmark Huggingface:…
2025 · getomni.ai
- 3IM
This is another one of my automate-my-life projects - I'm constantly asking the same question to different AIs since there's always the hope of getting a better answer somewhere else. Maybe ChatGPT's answer is too short, so I ask Perplexity. But I realize that's hallucinated, so I try Gemini. That answer sounds right, but I cross-reference with Claude just to make sure. This doesn't really apply to math/coding (where o1 or Gemini can probably one-shot an excellent response), but more to online search, where information is more fluid and there's no "right" search engine + text…
2024 · ithy.com
- 4
- 5
- 6AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 7MA
I've been exploring the (not so=) amazing potential of AI in coding and have compiled a list of tools. From AI-powered IDEs to code generators, this resource is my contribution to the community. I'm still on the fence about including txt2sql projects, as their functionality seems too basic to me. And I'm personally maintaining this, so your feedback is wellcome.
2025 · aicode.danvoronov.com
- 8CT
I've been building a tool that changes how LLM coding agents explore codebases, and I wanted to share it along with some early observations. Typically claude code globs directories, greps for patterns, and reads files with minimal guidance. It works in kind of the same way you'd learn to navigate a city by walking every street. You'll eventually build a mental map, but claude never does - at least not any that persists across different contexts. The Recursive Language Models paper from Zhang, Kraska, and Khattab at MIT CSAIL introduced a cleaner framing. Instead of cramming everything into…
Feb 2026 · github.com
- 9
Hi y'all. Been working on something that should've been made a long time ago imo. It compiles codebases into O(1) hashmaps that the agent queries to discover the structure of your code/answer questions/write code. It also does complete static analysis checks on any writes the agent makes. Don't take my word for it though. Here are the benchmarks: https://benzi.fly.dev/benchmark. on 2/20 tests, Claude Code (mostly Sonnet on one task) regressed or timed out. Benzi didn't because of course, it has a map it can query and not get lost in the sauce. On the other 18 it…
29d ago · benzi.fly.dev
- 10BY
we had hundreds of discussions with engineering leaders over the past few months, and everyone's trying to understand where they are in the AI journey. we collected all this data into a benchmark and built a free grader to let you know where you stand. you answer on a 1–5 scale (e.g., autonomy runs from "suggestions only" to "agents own multi-hour workflows across code, infra, and external systems") - takes about 5 minutes. https://agent-benchmarks.com/software-factory/ waiting for your results!
Jul 2026 · agent-benchmarks.com
- 11MD
We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…
2025 · github.com
- 12
- 13
Free master prompts for ChatGPT, Claude, Gemini & more
Mar 2026 · understandingai.net
- 14IG
Each AI has strengths - Claude reasons well, Gemini handles long context, Codex integrates with GitHub. But switching between them means losing context. Built HiveTechs: one workspace where Claude Code, Gemini CLI, Codex, DROID, and 7 others run in integrated terminals with shared memory. Also added consensus validation - 3 AIs analyze independently, 4th synthesizes. Real IDE with Monaco editor, Git, PTY terminals. Not a wrapper. Looking for feedback: hivetechs.io
Dec 2025 · hivetechs.io
- 15HW
A bunch of companies that I spoke to had their own claude & codex OTel dashboards that showed spend + seats per month. However, none of the dashboards actually analyzed how the engineers worked with the tools and if there were any areas for improvement! That's why I created https://www.promptster.ai. Managers get aggregate level view of code quality and how that ties with team workflows (nothing on a per-engineer level). While engineers get personalized coaching on how they can save tokens while keeping output high. We also have a tool built for individuals to test their local…
Jul 2026
- 16OD
The Problem "Vibing" with LLMs is often too shallow for complex logic, while writing full specifications is cognitively expensive and slow. We need a middle ground that mimics how human programmers gather context—scanning structure before diving into details. The Solution: Outline Driven Development (ODD) I've built a "batteries-included" kit for Gemini/Claude/Codex that uses AST analysis to understand code structure rather than just raw text. This relies on a hyper-optimized Rust toolchain (`ast-grep`, `ripgrep`, `jj`, etc.) to feed precise, structural context to the agent. 1. The…
Nov 2025 · github.com
- 17
- 18IM
Hello everyone, This past Saturday, I decided to make purchase on Cursor after hearing how good it is with Composer function that allow to generate code on fly, especially on Claude Sonnet (not an affiliate of the products, just curious!). To make the purchase worthwhile, I challenged myself to build a Google Sheets extension in just 24 hours that allows users to connect with any Language Model (LLM)—whether it’s ChatGPT, Claude, Groq, or Gemini. The Idea The idea was simple: a formula like =Claude(prompt, model) that lets users integrate AI anywhere in their sheets. The most impressive…
2024 · aisheeter.com
- 19HP
Hi HN! I'm building Hopsule. If you use AI coding tools like Cursor, Copilot, or Claude, you’ve probably seen this happen: The AI writes good code - but it ignores your architecture. It doesn’t know: - why you chose a specific pattern - which conventions your team agreed on - which decisions are already locked in So it falls back to generic patterns, outdated examples, or random GitHub training data. Over time this slowly breaks the consistency of the codebase. Most teams try to fix this with: - giant Markdown files - wiki pages - long prompts - Slack threads But those aren't…
Mar 2026
- 20
Compare GPT, Claude, Gemini & DeepSeek by cost & benchmark
11d ago · universalnest.com
- 21
- 22IB
Hi! My name is Herve Kom, a computer science student that is interested in learning new things everyday! As one of my graduation project, I have developed a Claude Code -like Coding CLI, but with enhancement for API Testing: - Auto-generate & run tests (unit, e2e, Playwright, CI/CD, etc.) - Say bye-bye to hallucinations with built-in MCP Server to let LLM directly read from API Docs - Adding Agent.md support for better context persistence across your whole codebase - Automatic bug & security scans (logic is kind of basic but works great!) - Vibes, I want it to feel less "enterprise" but…
2025 · github.com
- 23OS
We built an open-source library of 125 GTM (go-to-market) skills that plug into AI coding agents like Claude Code, Codex, and Cursor. With these skills an AI agent can automatically: - Find ICP leads from conference speakers, LinkedIn activity, or job boards - Generate personalized cold email sequences - Monitor competitor blogs, pricing pages, and hiring signals - Generate programmatic SEO pages from keyword lists - Track where your brand appears in ChatGPT, Perplexity, and Claude answers --- How skills work Each skill is a structured markdown file containing instructions, scripts, and tool…
Mar 2026 · github.com
- 24CB
I built a small benchmark to test CLI coding agents on blind bug detection. A challenger agent injects bugs and writes ground truth (`bugs.json`). A different reviewer agent audits the repo without seeing ground truth, and an LLM matcher scores bug-to-finding assignments. Current run: 50 repos, 150 challenges, 450 reviews, 2,603 injected bugs. Weighted detection: Claude 58.05%, Codex 37.84%, Gemini 27.81%. LLM-judge benchmarks are easy to get wrong, so I’d really appreciate critical feedback on benchmark fairness, scoring/matching methodology, and obvious failure modes I’m missing. Full…
Feb 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →