Alternatives
Products that do what Predict GPT-5 skills with a community AI benchmark does
I’m Andrew, co-founder of Recall. Over the past few days I’ve been building Predict, a playground where anyone can: - propose skills we should measure in language models—live examples include difficult math, memory-manipulation resistance, code generation, and empathy under bad news - write evals (graded prompts) for those skills - forecast which models will score highest once GPT-5 is released Why this exists Benchmarks leak into training data quickly; scores are unreliable and labs still declare progress. The prediction tool aims keeps the target moving by letting the crowd define both the…
- 1
- 2FC
Hi HN! I've found this visualization tool immensely helpful over the years for getting an intuition for how an LLM "sees" some piece of text, and with a bit of elbow grease decided to move all compute to client side so I could make it publicly available. I've found it particularly useful for - Understanding exactly how repetition and patterns affect a small LM's ability to predict correctly - Understanding different tokenization patterns and how it affects model output - Getting a general sense of how "hard" different prediction tasks are for GPT-style models Known problems (that I probably…
2023 · perplexity.vercel.app
- 3
- 4
- 5
- 6
- 7IR
2023 · sagittarius.greg.technology
- 8
- 9NT
What inspired this project today was watching this amazing video by 3Blue1Brown called "But what is a GPT?" on Youtube (https://www.youtube.com/watch?v=wjZofJX0v4M - I highly recommend watching it). I added it to the repo for reference. When it clicked in my head that "knowing a fact" is nearly synonymous with predicting a word (or series of words), I wanted to put it to the test, because it seemed so simple. I chose JavaScript because I can exploit the way it structures objects to aid in the modeling of language. For example: "I want to be at the beach", "I will do it later",…
2024 · github.com
- 10
- 11
- 12
- 13
- 14LT
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models. We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.” TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context. Our instantiations, TTT-Linear…
2024
- 15AL
Hi HN! We partnered with the Atlas team to build a tool called AI Predict [0] that allows anyone to ask any question about the future and get a thoroughly researched, AI-generated prediction on how likely it is to be true. How it works: Atlas replicated a Berkeley paper [1] that showed LLMs could make predictions as accurate as the crowd. We’re using a mix of models from OpenAI and Anthropic, with information retrieval powered by NewsCatcher [2]. The system is live and fully functional, though it might struggle with hyper-local questions outside of the public domain (e.g., “Will I have…
2024 · aipredict.fun
- 16BP
TLDR: Product developers need tools designed for them to build ML models. We’d love for you to try a demo of Mage without needing to sign up: https://www.mage.ai/onboarding My name is Tommy DANGerous (or Tommy Dang) and I’m the CEO and co-founder at Mage. I worked at Airbnb for over 5 years as a product developer building features for guests. Mage is a web-based tool for building, training, and deploying ML models that make predictions based off your data. Training and using ML models in production typically requires working knowledge of building data pipelines, algorithms,…
2022
- 17MD
We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…
2025 · github.com
- 18TA
2023 · github.com
- 19AS
We explored a novel method to gauge the significance of tokens in prompts given to large language models, without needing direct model access. Essentially, we just did an ablation study on the prompt using cosine similarity of the embeddings as the measure. We got surprisingly promising results when comparing this really simple approach to integrated gradients. Curious to hear thoughts from the community!
2023 · heatmap.demos.watchful.io
- 20ΤB
τ-Bench is an open benchmark for evaluating AI agents on grounded, multi-turn customer service tasks with verifiable outcomes. It's been great to see the community adopt it since launch — this is now the third iteration. With τ³-Bench, we're extending it to two new settings: knowledge-intensive retrieval and full-duplex voice. τ-Knowledge: agents must navigate ~700 interconnected policy documents to complete multi-step tasks. Best frontier model (GPT-5.2, high reasoning) hits ~25%. The surprising part: even when you hand the model the exact documents it needs, performance only reaches ~40%.…
Mar 2026
- 21
- 22CB
Hey HN, we're excited to share Cua-Bench ( https://github.com/trycua/cua ), an open-source framework for evaluating and training computer-use agents across different environments. Computer-use agents show massive performance variance across different UIs—an agent with 90% success on Windows 11 might drop to 9% on Windows XP for the same task. The problem is OS themes, browser versions, and UI variations that existing benchmarks don't capture. The existing benchmarks (OSWorld, Windows Agent Arena, AndroidWorld) were great but operated in silos—different harnesses,…
Jan 2026 · github.com
- 23IM
Also inspired by this HN submission: https://www.chiark.greenend.org.uk/~sgtatham/quasiblog/findl... The model is gpt-4o-mini-2024-07-18.
2024 · app4.hc11.org
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →