Alternatives
Products that do what Llm.report – Open-source logging and analytics for OpenAI does
Hi hacker news, My name is Dillion and I'm the creator of llm.report. A few months ago, I was frustrated by the lack of observability into the OpenAI API. All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product. So I ended up just building it myself. There are three parts to the platform: 1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API…
- 1LR
Hi! My OpenAI costs started going up recently and I hated how I couldn't figure out which model was costing me more? The OpenAI usage page (https://platform.openai.com/account/usage) gives a nice overall breakdown, but it doesn't go into detail about cost / model, token usage, and # requests. So I reverse engineered their API to make a better dashboard, and then made it free to use. Try it out and let me know what you think. Also please tell me what other features you wish existed here https://platform.openai.com/account/usage but doesn't yet.
2023 · llm.report
- 2
- 3
- 4
- 5MY
LLM observability is an absolute must-have for anyone running something in prod (or prod-like). While all the observability startups are great, you're essentially sending all your OpenAI usage history - prompts, generations, chats - to a random third party. So this script deploys a basic proxy in your Azure account, catches all incoming OpenAI requests, stores logs in your own resource group, and comes with visualizations premade (charts, timelines, chat history, cost estimation, etc). Thanks for any thoughts and feedback!
2023 · github.com
- 6
- 7
- 8
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 9
- 10
- 11
- 12
- 13IB
Hey HN, I've been working on something cool that I wanted to share with you all. It's called Viewpoint, an analytics tool for LLMs like OpenAI, Anthropic models, and Gemini. The idea came from the constant flood of new LLM models and the need to figure out which ones work best for my projects without breaking the bank. With viewpoint, I can track token usage, costs, latency(WIP), and traffic over time, making it easier to compare different models and see which ones perform best and save money. The tool works asynchronously, so it doesn't add any latency to your LLM requests, and you have…
2024 · viewpointhq.com
- 14IB
I’ve spent the last 2.5 months building a product that runs LLM-powered code reviews on my pull requests — and I just launched it. The tool is built specifically for solo developers. You install it on your repo, trigger a scan by creating a pull request, and it leaves structured review comments using OpenAI under the hood. Funnily enough, I used the dev version of this app to review its own pull requests while building it. It helped me spot bugs, simplify structure, and keep quality high — all with minimal need for another human in the loop. Things I want to try out in the next months : -…
2025 · codii.dev
- 15TO
I built TraceAIO, an open-source tool that prompts LLMs on your behalf and tells you whether ChatGPT, Perplexity, and Gemini mention your brand — and which competitors and sources show up instead. Yeah, this category smells a bit like a grift, same as early SEO. And I think over time it will become just SEO again, and become about good content. The tool just helps you monitor over time. It queries the browser products through real browser sessions, not APIs, runs on Docker, with an MCP server so you can query your own data through an LLM. No business model, Apache 2.0, self hosted. If you…
Jun 2026 · traceaio.org
- 16DT
Hello Folks, I am an indie developer, For the last 5 months I was freelancing by building GPT-powered chatbots. One of the challenges I faced was not having visibility into my OpenAI costs. This is why I have built a dashboard to get a complete breakdown of my OpenAI API usage through network calls from OpenAI. Please give it a try & help with feedback on how I can further improve the product,which can really add value and save some money. Note: 1)I found this API through the network logs of the billing screen. 2)The tool stores the OpenAI Key locally on the browser.
2023 · puddl.io
- 17DR
The first ever AI peer reviewed research article just got approved. It’s kinda crazy how advanced AI have come to replace researchers. I've just been using Deep Research on ChatGPT and Perplexity a lot to write and research complex technical reports for my boss. He loves the reports and it has decreased my workload a ton but I still have some frustrations with it. None of them provide an API that gets me the same quality of output you would with the applications. I wanted something with more control on the LLMs, swappable with the reasoning new models that came out. Not just prompt →…
2025 · github.com
- 18IB
2025 · interlify.com
- 19LO
Hey HN! I built Lumina – an open-source observability platform for AI/LLM applications. Self-host it in 5 minutes with Docker Compose, all features included. The Problem: I've been building LLM apps for the past year, and I kept running into the same issues: - LLM responses would randomly change after prompt tweaks, breaking things - Costs would spike unexpectedly (turns out a bug was hitting GPT-4 instead of 3.5) - No easy way to compare "before vs after" when testing prompt changes - Existing tools were either too expensive or missing features in free tiers What I Built: Lumina is…
Jan 2026 · github.com
- 20SA
Hi HN, A few weeks ago, we stealth launched SEC Insights on Product Hunt and were thrilled to reach the top 5! What is SEC Insights? It harnesses the Retrieval Augmented Generation (RAG) capabilities of LlamaIndex to provide answers about SEC 10-K & 10-Q documents. The enthusiastic reception we received on Product Hunt prompted us to open-source our project. Now, developers can use our project either as a reference or they can fork it entirely for their LLM-powered applications. We at LlamaIndex built this to help developers move their LLM app ideas from prototype to production. We're very…
2023 · github.com
- 21IS
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
2025
- 22LT
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com
- 23EA
A few months ago I was working on a flight search engine that would include pet transport costs (I know a few by hearth but storing them and make the calculations in the UI would be nice) While I was collecting pet pricing from several airlines I strugled to extract data in a common format without hallucinated values. That's when I thought: What if I use multiple LLMs and take the most common response to improve accuracy? This idea became this new project. You provide your documents, an SQLModel schema, an LLM provider, plus what you'd like to extract and Extrai does the rest. Including…
Nov 2025 · github.com
- 24TO
Hey Hackernews! We’ve launched our new AI-powered report builder and we’re eager to get feedback. We set this up with some demo retail data, but this can plug into any SQL database to provide customer reporting through natural language. I’ve seen this type of feature demo well on other platforms, but never actually work, so we’re putting this out into the world for anyone to actually try themselves. We’re leveraging OpenAI under the hood and have made a ton of improvements to the prompt engineering to configure the reports (OpenAI doesn’t actually write the underlying SQL). Click into a…
2025 · custom.explo.co
Ranked by how close each launch is in meaning, then by votes. Refine with a description →