Life & fun · December 11, 2023
I implemented evals metrics for LLMs that runs locally on your machine
In plain words
This is a local evaluation metrics tool for large language models that runs on users' own machines. It's designed for developers and researchers who need to assess LLM performance without relying on external services. The tool offers the advantage of privacy and offline functionality, allowing users to evaluate their models directly on their hardware rather than sending data to third-party platforms.
written from the facts on this page · September 2026
Does the same job
all alternatives →- FTFind the best local LLM for your hardware, ranked by benchmarksMay 2026 · github.com · ▲283
- LLLocalScore – Local LLM Benchmark2025 · localscore.ai · ▲124
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
- POPromptTools – open-source tools for evaluating LLMs and vector DBs2023 · github.com · ▲211
Hey HN! We’re Kevin and Steve. We’re building PromptTools (https://github.com/hegelai/prompttools): open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. Evaluating prompts, LLMs, and vector databases is a painful, time-consuming but necessary part of the product engineering process. Our tools allow engineers to do this in a lot less time. By “evaluating” we mean checking the quality of a model's response for a given use case, which is a combination of testing and benchmarking. As examples: - For generated…
- CLCreate LLM graders and run evals in JavaScript with one file2025 · github.com · ▲28
Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
More life & fun this month
the category →- TL
Life & fun · 10d ago · louisabraham.github.io
Articos▲385Launch with confidence, not gut instinct Discussion | Link
Life & fun · 12d ago · producthunt.com
Nex▲351Claude Cowork for high-volume GTM workflows Discussion | Link
Life & fun · 3d ago · producthunt.com
Photosynthesis fires two of your iPhone
Life & fun · 29d ago · photosynthesis.camera
Creatium Coach▲320Your multimedia mentor that takes you from mid to great Discussion | Link
Life & fun · 11d ago · producthunt.com
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 28d ago · solouno.io
Launched alongside, December 2023
the whole month →- BM
Hi HN! I’m proud to share that we have built a real 3rd party iMessage client for Android. We did it by reverse engineering the iMessage protocol and encryption system. It's available to download today (no waitlist): https://play.google.com/store/apps/details?id=com.beeper.ima and there's a technical writeup here: https://blog.beeper.com/p/how-beeper-mini-works. Unlike every other attempt to build an iMessage app for Android (including our first gen app), Beeper Mini does not use a Mac server relay in the cloud. The app connects directly to Apple…
Commerce · 2023 · beeper.com
- HA
Hey! I made Heynote entirely for my own use case. For many years, I always had an Emacs instance running with the scratch buffer open, even long after I had abandoned Emacs as my programming editor in favor of more recent IDE:s. The simplicity of having just one big scratch buffer appeals to me, but I still want to separate the different things I jot down somehow (without using tabs or similar). Previously, my solution was to insert a bunch of blank lines between the notes, but hitting C-A would still select the entire buffer. That's why I came up with the concept of "blocks", which turned…
Work · 2023 · heynote.com
FuseBase▲941Client collaboration platform for professional services
Dev tools · 2023 · thefusebase.com
Beep! 2.0▲871Fastest tool to assign and share visual tasks with your team
Work · 2023 · justbeepit.com
