LLM Thematic Generalization Benchmark

What it does

This benchmark tests whether large language models can infer a specific latent theme from a few examples, use anti-examples to reject the broader but wrong pattern, and then identify the one true match among close distractors. Models score all 8 candidates. We then measure whether the correct candidate is ranked first, and how highly it tends to be ranked overall. V2 uses 1,247 validated prompts and adds stricter ambiguity filtering plus harder evaluation slices. The previous published release is being archived as V1. The chart uses inverse-rank score , a higher-is-better transformation of the average rank assigned to the correct answer. It is easier to interpret than raw average rank and…from github.com

• 
  LLM Creative Story‑Writing Benchmark V32025 · github.com · ▲8

  This benchmark tests how well LLMs incorporate a set of 10 mandatory story elements (characters, objects, core concepts, attributes, motivations, etc.) in a short creative story - lechmazur/writing

• 
  Find the best local LLM for your hardware, ranked by benchmarksMay 2026 · github.com · ▲283

  Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm

• 
  LLM StatsOct 2025 · llm-stats.com · ▲293

  I was overspending on GPT-4o. It was really hard to compare different models I could switch to, so I built this LLM comparison tool. It shows leaderboards, pricing, and performance data across 100+ LLMs (including all major providers and open-source models). Key features: - Live pricing comparisons - Benchmark Scores (MMLU, HumanEval, GPQA, etc.) - Context length vs cost analysis - Speed/throughput tests across providers - Quality vs price visualizations - Open source (all data verifiable) Try it out: https://llmstats.com I'd like to know your opinion :) Tech stack: Next.js,…

• 
  LLM Deceptiveness and Gullibility Benchmark2024 · github.com · ▲7

  Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics. - lechmazur/deception

• 
  Deepmark AI2023 · github.com · ▲97

  LLM benchmarking tool for task-specific metrics on your data

• 
  Fully client-side GPT2 prediction visualizer2023 · perplexity.vercel.app · ▲153

  Hi HN! I've found this visualization tool immensely helpful over the years for getting an intuition for how an LLM "sees" some piece of text, and with a bit of elbow grease decided to move all compute to client side so I could make it publicly available. I've found it particularly useful for - Understanding exactly how repetition and patterns affect a small LM's ability to predict correctly - Understanding different tokenization patterns and how it affects model output - Getting a general sense of how "hard" different prediction tasks are for GPT-style models Known problems (that I probably…

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 18d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 18d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 20d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 21d ago · stillwet.art

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 30d ago · world.pixverse.video

• 
  Beam.win▲378

  Run AI agents on one Mac and use its windows on another

  AI · 4d ago · beam.win

• 
  I made an open-source laptop from scratch▲3,237

  Hello! I'm Byran. I spent the past ~6 months engineering a laptop from scratch. It's fully open-source on GH at: https://github.com/Hello9999901/laptop

  Dev tools · 2025 · byran.ee

• TI
  Tetris in a PDF▲1,289

  I realized that the PDF engines of modern desktop browsers (PDFium and PDF.js) support JavaScript with enough I/O primitives to make a basic game like Tetris. It was a bit tricky to find a union of features that work in both engines, but in the end it turns out that showing/hiding annotation "fields" works well to make monochrome pixels, and keyboard input can be achieved by typing in a text input box. All in all it's quite janky but a nice reminder of how general purpose PDF scripting can be. The linked PDF is all ASCII so you can just open it in a text editor, or have a look at…

  Life & fun · 2025 · th0mas.nl

• 
  21st.dev▲1,211

  I originally built this app to organize and reuse interesting React + Tailwind components I came across on Twitter, inspired by the shadcn/ui approach. Later, a friend started contributing their own components, and it got me thinking—this could grow into something bigger, like an NPM for design engineers to share and use components. Would love to hear your thoughts!

  Dev tools · 2025 · 21st.dev

• 
  JoggAI 2.0: AI Avatar Generator▲967

  Create lifelike, personalized AI avatars from text prompts

  AI · 2025 · jogg.ai

• 
  Lovable + Builder.io▲873

  Turn Figma designs into full stack apps with AI

  Dev tools · 2025 · figma.com

• 
  Cs16.css – CSS library based on Counter Strike 1.6 UI▲819

  CSS library based on Counter Strike 1.6 UI.

  Life & fun · 2025 · cs16.samke.me