Paramount – Human Evals of AI Customer Support
Hey HN, Hakim here from Fini (YC S22), a startup focused on providing automated customer support bots for enterprises that have a high volume of support requests. Today, one of the largest use cases of LLMs is for the purpose of automating support. As the space has evolved over the past year, there has subsequently been a need for evaluations of LLM outputs - and a sea of LLM Evals packages have been released. "LLM evals" refer to the evaluation of large language models, assessing how well these AI systems understand and generate human-like text. These packages have recently relied on…
Paramount – Human Evals of AI Customer Support launched on June 13, 2024 with 71 votes, #678 of 4,672 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Paramount is a human evaluation platform for AI customer support systems built by Fini, a startup specializing in automated support bots for enterprises. It addresses the limitations of automatic evaluations by incorporating human review to achieve the higher accuracy thresholds required by large-scale support operations. The tool is designed for companies deploying LLM-based customer support that need reliable quality assessment beyond algorithmic scoring.
written from the facts on this page · September 2026
From the sources
Agent accuracy measurements for LLMs. Contribute to ask-fini/paramount development by creating an account on GitHub.
In the maker’s words, at launch
Hey HN, Hakim here from Fini (YC S22), a startup focused on providing automated customer support bots for enterprises that have a high volume of support requests. Today, one of the largest use cases of LLMs is for the purpose of automating support. As the space has evolved over the past year, there has subsequently…
Does a similar job
all alternatives →
Paramount is an OSS package that captures expert feedback on LLM chats2024 · github.com · ▲9Hey HN, Hakim here from Fini (YC S22). We've seen first hand how AI chat projects pan out, and so have released an OSS library to ensure the industry gets more tools for improving outcomes. Many AI chat projects are scrapped due to persistent inaccuracies in LLM responses. Paramount is an open-source Python package designed to bridge the gap between LLM-generated and ideal responses by incorporating expert feedback directly into the evaluation process. It provides a robust framework for recording LLM function outputs (ground truth data) and facilitates agent evaluations, reducing the time to…
Create LLM graders and run evals in JavaScript with one file2025 · github.com · ▲28Hi HN! Run it: OPENROUTER_API_KEY="sk" npx bff-eval --demo We built a tool to help people take LLM outputs and easily grade them / eval them to know how good an assistant response is. We've built a number of LLM apps, and while we could ship decent tech demos, we were disappointed with how they'd perform over time. We worked with a few companies who had the same problem, and found out scientifically building prompts and evals is far from a solved problem... writing these things feels more like directing a play than coding. Inspired by Anthropic's constitutional ai concepts, and amazing…
- TLTiny LLMs – Browser-based private AI models for a wide array of tasks2023 · tinyllms.vercel.app · ▲142
Humans vs LLMs2025 · aipageready.com · ▲138Hi HN, We just launched [AI Page Ready](https://aipageready.com) — a tool to check if your web pages are discoverable and understandable by AI tools like ChatGPT, Claude, Gemini, and Perplexity. It runs 40+ checks across six areas: Discoverability & Crawlability, Structured Data & Semantics, Content Quality, Answer-Ready Formatting, Accessibility & Rendering, and Deep AI LLM-specific checks. Would love your feedback — especially if you work in eCommerce, SEO, content, SaaS, or are trying to get traffic from AI tools. (Also open to ideas on what else to add to the tool!)
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, June 2024
the whole month →
La Growth Machine▲1,216Create personalized, multi-channel conversations at scale
AI · 2024 · lagrowthmachine.com
PyjamaHR▲1,132Hiring on autopilot. The AI applicant tracking system (ATS).
Work · 2024 · pyjamahr.com