Alternatives
Products that do what LLMadness – March Madness Model Evals does
I wanted to play around with the non-coding agentic capabilities of the top LLMs so I built a model eval predicting the March Madness bracket. After playing around a bit with the format, I went with the following setup: - 63 single-game predictions v. full one-shot bracket - Maxed out at 10 tool calls per game - Upset-specific instruction in the system prompt - Exponential scoring by round (1, 2, 4, 8, 16, 32) There were some interesting learnings: - Unsurprisingly, most brackets are close to chalk. Very few significant upsets were predicted. - There was a HUGE cost and token disparity with…
- 1AM
2019 · github.com
- 2UP
Hi! I made this as a side project this week to submit a bracket for our friends' NFL playoff bracket competition. It is all html and vanilla JS, runs directly in the browser, and is pretty fun to take for a spin. Obviously it is far from polished - that might have to wait for next year!
2023 · nflbracket.xyz
- 3
- 4MM
2014 · codersbracket.com
- 5
- 6CT
2018 · cryptofinalfour.com
- 7PP
I've been working on applying LLMs to long-context, verifiable problems over the past year, and today I'm releasing a benchmark of 62,000 pencil puzzles across 94 types (sudoku, nonori, slitherlink, etc.). The benchmark also allows for intermediate checks /rule breaks for all varieties at any step. I tested 51 models against a subset (300 puzzles) in two modes: single-shot (output the full solution) and agentic (iterate with verifier feedback). Some results: - Best model (GPT 5.2@xhigh) solves 56%. (~ half the puzzles are unsolved by any model) - Agentic solves average 29 turns. The…
Mar 2026 · ppbench.com
- 8TM
2012 · otherscreen.com
- 9GA
I wanted to learn more about RAG implementations, so I built something to solve the constant digging through manuals whenever we play a game. It's fairly simplistic, but actually has worked pretty well for some of these conflicts. Everythings Open Source on GitHub if you're curious (or have ideas), and I'd love to hear feedback from fellow boardgamers!
2024 · gamegame.ai
- 10SE
Hey HN! I built self-driving sim and eval at Waymo. Now I’m building Scorecard to bring that approach to agent eval: reproducible, automated scoring for AI. Scorecard lets you: - Run LLM-as-judge evals on agent workflows: test tool usage, multi-step reasoning, and task completion in CI/CD or in a playground. - Debug failures with OpenTelemetry traces: see which tool failed, why your agent looped, and where reasoning went wrong. - Collaborate on datasets, simulated agents, and evaluation metrics. Try it out → https://app.scorecard.io (free tier, no payment required!) Docs →…
Oct 2025 · docs.scorecard.io
- 11DN
Hey everyone, I’m an NBA fan and Python dev, and I recently built DeepShot — a machine learning model that predicts NBA game outcomes with about 71% accuracy based on historical stats and rolling performance metrics (EWMA). It features: Real NBA data from Basketball Reference Exponentially Weighted Moving Averages to track momentum Interactive NiceGUI interface with team comparison and predictions Full Python stack and open-source (MIT license) Here’s the GitHub repo: https://github.com/saccofrancesco/deepshot And if you like it, here’s my Buy Me a Coffee:…
2025 · github.com
- 12MM
2015 · github.com
- 13RE
Recent academic work ([1], [2]) has suggested that LLMs can effectively simulate different Internet subpopulations. For example, you may ask ChatGPT to emulate being a high school teacher explaining Newton’s laws of physics. Building upon this, we created Roundtable, a platform that uses LLMs to predict how people will respond to any arbitrary survey question. To do so, we needed to first reduce bias arising from GPT’s training procedure. Because these models are primarily trained on Internet data, they can be heavily skewed towards the demographics of heavy Internet users (e.g.,…
2023 · roundtable.ai
- 14GO
I couldn't find a nice worldcup bracket visualizer out there so ended up building one. It's nicer than the ones I saw out there. There is no login, no ads, runs client side, uses localStorage and all the state is encoded in the URL itself and you can share it. The calendar feature is nice. I wanted to build my own schedule for the games I was interested in. There is full calendar that you can subscribe to as well, that gets updated as the fixtures get decided. I also wanted to be able to interactively change things and visualize the whole knockout stage. All the state is in the URL fragment.…
Jun 2026 · groupofdeath.gg
- 15IM
Hi everyone, I just saw a vibe coded game on HN, and thought maybe I should post about this here. I'm organizing a vibe coding game dev competition called Vibe Jam. Last year we did it too and there was 1000+ games submitted. This year the deadline is May 1 and you can submit your games until then. There's $35,000 in prizes with the Gold prize being $20,000. Let me know what you think! -Pieter
Apr 2026 · vibej.am
- 16WM
We wanted to test if a smaller model like GPT-4.1-mini could beat its bigger brother 4.1 at the game Tic-Tac-Toe using only context engineering. We put them in a 100-game tournament. For the smaller model, we gave it a few examples of winning moves from past games right before it made its own move. The results were clear. Without the examples, the smaller model struggled against GPT-4.1. With the examples, its effectiveness increased by nearly 200%, and it consistently won. It's a simple demonstration, but it shows that a smaller, faster model with good, timely examples can outperform a more…
2025 · github.com
- 17GV
Hey HN, I just updated my project that compares some LLMs. It uses your prompt for all the models and runs at the same time. You can see the results being generated in real-time and decide what's the best for your use case. I'm open to any suggestions and feedback. Thanks!
2024 · geminivsgpt.com
- 18IL
LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more. There are thousands of permutations to try. We need to be able to easily experiment with these…
2024 · palico.ai
- 19IB
Hey HN, I've been working on something cool that I wanted to share with you all. It's called Viewpoint, an analytics tool for LLMs like OpenAI, Anthropic models, and Gemini. The idea came from the constant flood of new LLM models and the need to figure out which ones work best for my projects without breaking the bank. With viewpoint, I can track token usage, costs, latency(WIP), and traffic over time, making it easier to compare different models and see which ones perform best and save money. The tool works asynchronously, so it doesn't add any latency to your LLM requests, and you have…
2024 · viewpointhq.com
- 20AD
Ever wish you could get the best arguments for both sides of a debate? I built an AI-powered debate platform that pits language models against each other on controversial topics. Each AI is randomly assigned a side (pro/con). You vote before and after to see if you were persuaded. Most content today presents lopsided arguments. They provide strong points for one side, weak ones for the other. This project aims to surface the strongest arguments from both sides, using LLMs to simulate a fair debate. With enough usage, I want to use it to benchmark LLMs. My hypothesis is that randomly…
2025 · bot-bicker.vercel.app
- 21TN
Hi guys, I’m excited to share an update on ReproModel, an open-source toolbox designed to streamline the testing and reproduction of machine learning models. I, like many of you, have really struggled with benchmarking and comparing models, from missing code, to opaque experiment parameters slowing the process. I decided to take matters into my own hands, and created a mini-toolbox in my free time to streamline the process. The goal is to reduce the time and effort spent on replicating experiments, enabling researchers to focus on innovation rather than setup. Knowing this task is not an…
2024 · github.com
- 22NL
Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction. Here is the interactive demo: https://labs.refuel.ai/playground. Pretty fun to play with!
2023
- 23ST
Hi HN! I’m a founder at Nextmv (YC 20) [1] We’ve been building out optimization algorithm testing capabilities (acceptance tests, etc.) and just shipped our first pass at shadow testing [2, 3]. In our experience, tools like shadow testing save time and build confidence in decision models, but tools also take time to build and maintain. We’ve seen shadow testing tools in the machine learning and MLOps space [4], but not so much in the operations research community. A lot of folks here [5] seem experienced with optimization models and we’d love to have your feedback! What do you like? What…
2023 · nextmv.io
- 24AA
An all-in-one blog for learning LLM ins and outs: tokenize, attention, PE, and more Project I've been diving deep into the internals of Large Language Models (LLMs) and started documenting my findings. My blog covers topics like: Tokenization techniques (e.g., BBPE) Attention mechanism (e.g. MHA, MQA, MLA) Positional encoding and extrapolation (e.g. RoPE, NTK-aware interpolation, YaRN) Architecture details of models like QWen, LLaMA Training methods including SFT and Reinforcement Learning If you're interested in the nuts and bolts of LLMs, feel free to check it out:…
2025 · comfyai.app
Ranked by how close each launch is in meaning, then by votes. Refine with a description →