Phospho – Text Analytics for LLM Apps (Posthog for Prompts)
Hello HN! Pierre and Paul here. We are building an open source text analytics tool for user inputs and LLM app outputs The repo is https://github.com/phospho-app/phospho and landing is https://phospho.ai Most people building with LLMs today don’t have quantified evaluation and usage metrics on the interactions between users and their product. The only solution is to read every message (or a sample) to get a sense of what is going on. You can't improve your product without understanding who your users are and how they are using it. Nobody would launch a website…
In plain words
Phospho is an open-source text analytics tool designed for developers building applications with large language models. It analyzes user inputs and LLM outputs to provide quantified metrics and insights on user interactions, similar to how traditional website analytics work. The tool allows developers to log messages and set up semantic events to understand user behavior and product usage patterns, enabling data-driven improvements to LLM-based applications.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hello HN! Pierre and Paul here. We are building an open source text analytics tool for user inputs and LLM app outputs The repo is https://github.com/phospho-app/phospho and landing is https://phospho.ai Most people building with LLMs today don’t have quantified evaluation and usage metrics on the interactions between users and their product. The only solution is to read every message (or a sample) to get a sense of what is going on. You can't improve your product without understanding who your users are and how they are using it. Nobody would launch a website without standard analytics today; the same principle should apply to LLM products. We made phospho to analyze the large amounts of text from user inputs and LLM app outputs, and give you quantified and actionable insights. You first log messages and set up semantic events. Eg: “user is talking about sports”, “assistant didn’t quote the source”. We then run asynchronous jobs to detect if events are present in the text or not. To do so, we use GPT-4 for the first few events, and then downsize to smaller fine-tuned models (cheaper & faster). It works with any LLM provider (OpenAI, Mistral, Ollama…). No proxy, no monkey patch, and no OpenAI key needed. You can link phospho to your users’ feedback, and even use the platform to annotate some messages yourself. This helps you design step by step a custom evaluation pipeline that runs automatically, fits your needs, and enables you to iterate. Results are available in dashboards, as dataframes, or via API. You can also directly leverage the events in your app to trigger actions in real-time with the API or via webhooks. Deploy everything with Docker, or use the hosted cloud version. We have Python/Javascript SDK and an API. License is Apache 2.0. Give it a spin and see where we’re at: https://github.com/phospho-app/phospho We’re interested in both feature requests and roasts. Let us know what you think!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, March 2024
the whole month →
- 3Y
Life & fun · 2024 · github.com
Microlaunch▲1,116Launch and get feedback on both the idea and product
Dev tools · 2024 · microlaunch.net
