Evaluating LLMs on creative writing via reader usage, not benchmarks
Hey HN! I'd love to get some people to mess around with a little side project I built to teach myself DSPy! I've been a big fan of reading fiction + webnovels for a while now, and have always been curious about two things: how can LLMs iteratively learn to write better based on reader feedback, and which LLMs are actually best at creative writing (research benchmarks are cool, but don't necessarily translate to real-world usage). That's exactly why I built narrator.sh! The platform takes in a user input for a novel idea, then generates serialized fiction chapter-by-chapter by using DSPy to…
In plain words
Narrator.sh is a platform that generates serialized fiction chapter-by-chapter based on user story ideas. It uses DSPy to optimize LLM writing output by incorporating real reader feedback across chapters, employing chain-of-thought reasoning and reward functions to improve subsequent writing. The tool is designed for readers and writers interested in exploring which LLMs perform best at creative writing in practical settings, moving beyond traditional research benchmarks to evaluate performance through actual usage and reader preferences.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN! I'd love to get some people to mess around with a little side project I built to teach myself DSPy! I've been a big fan of reading fiction + webnovels for a while now, and have always been curious about two things: how can LLMs iteratively learn to write better based on reader feedback, and which LLMs are actually best at creative writing (research benchmarks are cool, but don't necessarily translate to real-world usage). That's exactly why I built narrator.sh! The platform takes in a user input for a novel idea, then generates serialized fiction chapter-by-chapter by using DSPy to optimize the writing based on real reader feedback. I'm using CoT and parallel modules to break down the writing task, refine modules + LLM-as-a-judge for reward functions, and the SIMBA optimizer to recompile user ratings from previous chapters to improve subsequent ones. Instead of synthetic benchmarks, I track real reader metrics: time spent reading, ratings, bookmarks, comments, and return visits. This creates a leaderboard of which models actually write engaging fiction that people want to finish. Right now the closest evals for creative writing LLMs come from the author perspective (OpenRouter's usage data for tools like Novelcrafter). But ultimately readers decide what's good, not authors. You can try it at https://narrator.sh. Here's the current leaderboard: https://narrator.sh/llm-leaderboard (it's a bit bare right now b/c there's not that many users haha) (Fair warning: there's some adult content since I posted on Reddit for beta testers and people got creative with prompts. I'm working on diversifying the content!)
More work this month
the category →
The app store for voice native apps that lives in your notch
Work · 28d ago · voiceos.com
The New Calendly▲211Handle all of the work before, during, and after meetings
Work · 17d ago · calendly.com
Launched alongside, August 2025
the whole month →
- IS
I built the world's most impractical 1000-pixel display and anyone in the world can draw on it. It draws a single pixel at a time and takes 30-60 minutes to complete a single image. Anyone can participate in the project by voting for the next image to be drawn, and submitting images. https://kilopx.com/
Work · 2025 · benholmen.com
- KT
Kitten TTS is an open-source series of tiny and expressive text-to-speech models for on-device applications. We are excited to launch a preview of our smallest model, which is less than 25 MB. This model has 15M parameters. This release supports English text-to-speech applications in eight voices: four male and four female. The model is quantized to int8 + fp16, and it uses onnx for runtime. The model is designed to run literally anywhere eg. raspberry pi, low-end smartphones, wearables, browsers etc. No GPU required! We're releasing this to give early users a sense of the latency and voices…
Dev tools · 2025 · github.com
- IW
I was wondering how I can arrange objects along a spherical helix path, and read some articles on it. I ended up learning about parametric equations again, and make this visualization to document what I learned: https://visualrambling.space/moving-objects-in-3d/ feel free to visit and let me know what you think!
Life & fun · 2025 · visualrambling.space
- TC
For HTML Day 2025 [1], I made a web service that displays the current sky at your approximate location as a CSS gradient. Colours are simulated on-demand using atmospheric absorption and scattering coefficients. Updates every minute, without the use of client-side JavaScript. Source code and additional information is available on GitHub: https://github.com/dnlzro/horizon [1] https://html.energy/html-day/2025/index.html
Dev tools · 2025 · sky.dlazaro.ca