LLM Creative Story‑Writing Benchmark V3
This benchmark tests how well LLMs incorporate a set of 10 mandatory story elements (characters, objects, core concepts, attributes, motivations, etc.) in a short creative story - lechmazur/writing
LLM Creative Story‑Writing Benchmark V3 launched on September 10, 2025 with 8 votes, #1,468 of 8,160 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
This benchmark compares short stories written to the same constrained creative briefs. Separate evaluator models read matched story pairs and choose which one is better. Those choices are combined into a relative comparison score. Higher scores mean stronger performance against the other models tested. Scores are relative, not grades: zero is near the middle of this comparison set, and overlapping uncertainty ranges can indicate similarly rated models. Read each cell by row. Red means the row model performed better, blue means the column model performed better, and grey means the models were not directly compared. Near-white cells indicate close results. Both axes follow the leaderboard…from github.com
Does a similar job
all alternatives →
Mapping LLM Style and Range in Flash Fiction2025 · github.com · ▲7Documents the style side of the short-story Creative Writing LLM benchmark: we generated many short stories with a range of LLMs, then analyzed those stories for stylistic fingerprints and within-model diversity. This study focuses on how models write, how their outputs differ, and how varied each m
LLM Divergent Thinking Creativity Benchmark2024 · github.com · ▲8LLM Divergent Thinking Creativity Benchmark. LLMs generate 25 unique words that start with a given letter with no connections to each other or to 50 initial random words. - lechmazur/divergent
- ELEvaluating LLMs on creative writing via reader usage, not benchmarks2025 · narrator.sh · ▲36
Hey HN! I'd love to get some people to mess around with a little side project I built to teach myself DSPy! I've been a big fan of reading fiction + webnovels for a while now, and have always been curious about two things: how can LLMs iteratively learn to write better based on reader feedback, and which LLMs are actually best at creative writing (research benchmarks are cool, but don't necessarily translate to real-world usage). That's exactly why I built narrator.sh! The platform takes in a user input for a novel idea, then generates serialized fiction chapter-by-chapter by using DSPy to…
LLM Thematic Generalization Benchmark2025 · github.com · ▲6Thematic Generalization Benchmark: measures how effectively various LLMs can infer a narrow or specific "theme" (category/rule) from a small set of examples and anti-examples, then detect which item truly fits that theme among a collection of misleading candidates. - lechmazur/generalization


More ai this month
the category →



Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video

Source: Show HN post ↗
Launched alongside, September 2025
the whole month →

The site hit #1 on HN about a year ago. Got enough traffic that I turned it into a proper API. What's there: - REST API — 35k products, manually filterable by category, free key, 60 req/min - CLI — npx anycrap random -c food - faker.js plugin — bundled, no API call, works offline - HuggingFace dataset — full 35k rows for ML/training https://anycrap.shop/developers
AI · 2025 · anycrap.shop

I made a built-from scratch Wayland Compositor to display any GUI app* in the terminal! I think there is a lot of unexplored potential in custom Wayland compositors, a lot of really cool things you can embed existing applications into! So, I started with embedding apps into the terminal because that is the easiest input/output (output is just utf-8 and I use the great `chafa` library for that, and I just read from stdin for the input). If you have any other ideas for cool Wayland compositors, let me know. I purposedly wrote 80% the app in Typescript to appeal to the most developers and…
Dev tools · 2025 · github.com
- IR
Years ago I stumbled across a basic version of this concept and it stuck with me. I knew if I was ever going to take on such a project, it would need to be flawless, but without coding experience it was just another idea that would never happen. By the end of 2024, as AI coding tools exploded everywhere, I finally had a way to make it real. I started from zero knowledge and spent months collaborating with AI agents as a learning experience. Every pixel and every function went through me. The AI translated what I asked for into code, but every decision was human. I didn't use existing OS…
AI · 2025 · mitchivin.com

