Inferbench

What it does

InferBench is a vendor-neutral CLI tool that benchmarks local LLM inference engines like omlx and llama.cpp directly on your own hardware. Instead of relying on external metrics, it reports real, measured tokens per second. InferBench auto-detects engines, runs a fixed prompt set, and recommends the fastest configuration for your setup. It is self-hosted and Apache 2.0 licensed.

Benchmark local LLM inference engines (llama.cpp, omlx) on your own hardware with real tokens/sec, not borrowed numbers - RudrenduPaul/InferBench

• 
  Inferock BenchAug 2026 · github.com · ▲295

  Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts. - inferock/inferock-bench

• 
  Inferock22d ago · inferock.ai · ▲263

  Serverless inference, with credits back for failed calls.

• 
  We made our own inference engine for Apple Silicon2025 · github.com · ▲186

  We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.

• 
  PinchBenchMar 2026 · pinchbench.com · ▲305

  Find the best AI model for your OpenClaw

• 
  LocalScore – Local LLM Benchmark2025 · localscore.ai · ▲124

  Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…

• AT
  A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)2025 · llmapitest.com · ▲55

  I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 17d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 16d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 18d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 20d ago · stillwet.art

• 
  Onepin▲384

  AI voiceover with every name, price and date said right

  AI · 30d ago · onepin.ai

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 28d ago · world.pixverse.video

• TL
  The load-bearing vocabulary of Claude▲710

  Life & fun · Aug 2026 · louisabraham.github.io

• 
  Hey Noah▲656

  A proactive AI executive assistant for founders

  AI · Aug 2026 · heynoah.io

• 
  Clipto MCP▲655

  Hey HN, We recently built Clipto. It’s a tool that lets you search over terabytes of video, audio, and images on your computer, without relying on the cloud. Motivation: we probably all had this similar experience, we know a moment exists in a video or audio, but finding it takes hours scrubbing the timeline. You can send all the media to process in the cloud, but it’s slow, expensive and raises privacy concerns. So we decided to build our own on-device media search engine. How it works (high level): 1. We ingest video, audio, image; normalize formats via ffmpeg; run content analysis to…

  Work · Aug 2026 · clipto.com

• 
  Simple algorithm and color space to generate diverse skin tones▲627

  Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…

  Life & fun · Aug 2026 · toneyalexander.github.io

• 
  AdAnt AI▲620

  Claude for viral, high-converting social ads

  AI · Aug 2026 · adant.ai

• 
  Astute▲606

  Automate your B2B brand going viral, with new media creators

  AI · Aug 2026 · company-app.joinastute.com