Alternatives
Products that do what Open-source LLM observability tool for lazy devs does
Hi HN, we’re Dylan and Matthew, building sublingual (https://github.com/sublingual-ai/sublingual), an open-source LLM observability tool you can use with zero code changes. As developers focused on iterating and building features as fast as possible, we felt observability would’ve been a helpful tool to have, but we found existing solutions had too much overhead to set up. So we gave ourselves the challenge of building an observability tool that you can integrate without changing a single line of code in your project. How it works Run your python application as usual with…
- 1
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 2
- 3
- 4
- 5
- 6
- 7LO
Hey HN, Ola and Karthik here. We are working on Langtrace(https://github.com/Scale3-Labs/langtrace), an open source, open telemetry based SDK and monitoring/evaluations client for LLM based applications. The SDK generates OTEL standard spans and traces for popular LLMs like OpenAI, Anthropic and Cohere, popular frameworks like Langchain and LlamaIndex and vectorDBs like ChromaDB and Pinecone. The LLM monitoring/evaluations space has seen a number of products off late, both open source and closed source ones. But, a couple of things we have observed are: lack of…
2024
- 8MY
LLM observability is an absolute must-have for anyone running something in prod (or prod-like). While all the observability startups are great, you're essentially sending all your OpenAI usage history - prompts, generations, chats - to a random third party. So this script deploys a basic proxy in your Azure account, catches all incoming OpenAI requests, stores logs in your own resource group, and comes with visualizations premade (charts, timelines, chat history, cost estimation, etc). Thanks for any thoughts and feedback!
2023 · github.com
- 9LO
Hey HN! I built Lumina – an open-source observability platform for AI/LLM applications. Self-host it in 5 minutes with Docker Compose, all features included. The Problem: I've been building LLM apps for the past year, and I kept running into the same issues: - LLM responses would randomly change after prompt tweaks, breaking things - Costs would spike unexpectedly (turns out a bug was hitting GPT-4 instead of 3.5) - No easy way to compare "before vs after" when testing prompt changes - Existing tools were either too expensive or missing features in free tiers What I Built: Lumina is…
Jan 2026 · github.com
- 10CR
hi everyone. how does moving llm call prompts and output structure definitions away from code into configuration land sound? would you use something like this if it was stable and well documented enough? please don't hold back the criticism. i appreciate all feedback (constructive & otherwise).
2024 · github.com
- 11EL
Hey HN! I built Experiment to solve a common frustration in LLM development: the lack of proper tools for prompt engineering experimentation. Here's what makes it different: Key Features: - Load and edit chat completion logs from CSV files - Fork and modify specific conversation entries - Run inference via Anthropic, Mistral, and OpenAI - Define custom tools using JSONSchema format - Visual tool usage analysis with collapsible, sorted key-value pairs - Full mobile support and available as installable PWA Technical Highlights: - Built with React using custom isomorphic architecture -…
2025 · github.com
- 12LI
2018 · languagemodels.io
- 13DF
Hi HN, We’re the team at MyDecisive.ai, and today we’re giving developers a peek at Octant — point-and-click control and visibility for your OpenTelemetry. You've likely felt the pain of the "observability tax," especially if you manage K8S clusters. The modern standard is to instrument everything with OpenTelemetry, but piping all those rich OTLP logs, metrics, and traces straight to a SaaS vendor (Datadog, Splunk, Honeycomb) gets expensive fast. You end up paying massive ingestion and storage costs for noisy, low-value data just so it's searchable when something breaks. With Octant you get…
Jun 2026
- 14TA
Hi HN, TamedTable is an LLM harness for data ETL. And yes, it was developed using AI, meaning you can take the entire specification and recreate it to your desires: https://github.com/ZSvedic/TamedTable
Aug 2026 · tamedtable.com
- 15RO
Hi HN! RΞASON is an OSS Typescript framework for developing LLM apps that uses Typescript's interfaces to get structured output from an LLM. While there are other TS LLM frameworks, I think RΞASON fills a unique space in the market: it's laser-focused on only three areas and, most importantly, actively stays away from pre-made prompting & retrieval. I've been in the LLM space since GPT-3 originally came out, and I've always had problems with other frameworks, such as LangChain. I dislike that they focus a ton on out-of-the-box prompting & pre-made agents — I, as the dev, should be the one in…
2023 · github.com
- 16LB
Hello everyone. I built an AI-based toolset to help me with language learning. I wanted to be able to easily generate very specific study content and get rapid feedback on my writing. Unlike most language apps, it doesn’t actually try to teach you a language. Instead, it’s a collection of tools for people at an intermediate level who already have a learning process It’s particularly great for Anki users. There a demo video on the login page, and I set up anonymous auth for people who want to test it without creating an account. Feedback and bug reports welcome.
2025 · drillapp.xyz
- 17SD
Hey HN! After spending way too many nights debugging flaky AI tests, I built SteadyText. It's a simple python library for deterministic llm generations and embeddings. We use it in production for: - Testing our AI features (zero flakes in 3 months) - CLI tools that need consistent outputs - Reproducible documentation examples It's not for creative tasks - this is specifically for when you need AI to be boring and predictable. Think of it as the opposite of ChatGPT. The coolest part? It includes a Postgres extension. You can now do: SELECT steadytext_generate('explain this query: ...'); And…
2025 · steadytext.julep.ai
- 18LA
Hey HN, I'm sure many of you have encountered statically-typed codebases so large and complicated that your code editors freeze, lag, become unresponsive, and generally struggle. Debugging a slow editor is involved and usually an unwelcome distraction. In many cases, slowness in code editors comes from language servers, which are external programs that provide language features (e.g. go-to-definition, diagnostics, type hints). Examples of developer frustrations: [1] and [2]. At a previous company, we were concerned about growing internal frustrations from editors bottle-necked by slow…
2025 · github.com
- 19LF
Hey HN, I built SWE-Kit, LLM toolkit (Function callable tools) which makes building agents specialised in coding like Devin very easy. I noticed a typical pattern while building local agents: creating & perfecting LLM tools to interact with system or codebase was the repeated and time-consuming. We created a layer that simplifies building agents that can interact with code, file system, git, shell and allows you to quickly solve for a wide variety of coding agent use cases. Aren’t there open coding agents already? Well, yes, but most folks would want to solve their specific use case like a…
2024 · swekit.dev
- 20OS
I'm launching Linguist Translate, an open-source, full-featured translation solution with an embedded offline translator based on the Bergamot Project created by Mozilla. Site: https://linguister.io GitHub: https://github.com/translate-tools/linguist Today, Linguist is launched on ProductHunt. Check it out: https://www.producthunt.com/posts/linguist-translate Linguist is not just a wrapper over Google Translator like many other extensions. You can use any translation service with Linguist, thanks to custom translators! You may even deploy any…
2024
- 21PA
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
2024 · promptl.ai
- 22LR
Hi hacker news, My name is Dillion and I'm the creator of llm.report. A few months ago, I was frustrated by the lack of observability into the OpenAI API. All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product. So I ended up just building it myself. There are three parts to the platform: 1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API…
2023 · github.com
- 23
I find LLM interpretability extremely interesting and wanted to create a minimal repo for: - SAE training - Automatic feature interpretation - Visualizing features and running interventions through a GUI You can try it here: https://nanointerpret.pages.dev/ Or check the repo: https://github.com/Belluxx/nanointerpret
10d ago · nanointerpret.pages.dev
Ranked by how close each launch is in meaning, then by votes. Refine with a description →