Alternatives
Products that do what PromptMetrics - EU LLM Observability does
EU-Native LLM Observability. Stop Flying Blind on AI Spend.
- 1OO
Hey HN, Nir, Gal and Tomer here. We’re open-sourcing a set of extensions we’ve built on top of OpenTelemetry that provide visibility into LLM applications - whether it be prompts, vector DBs and more. Here’s the repo: https://github.com/traceloop/openllmetry. There’s already a decent number of tools for LLM observability, some open-source and some not. But what we found was missing for all of them is that they were closed-protocol by design, vendor-locking you to use their observability platform or their proprietary framework for running your LLMs. It’s still early in the…
2023 · github.com
- 2
- 3
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 4
- 5
- 6
- 7
- 8
- 9
- 10YD
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com
- 11
- 12OP
Open Prompts is the dataset used to build krea.ai. The data comes from the Stability AI Discord and includes around 10M images from 2M prompts. You can use it for creating semantic search engines of prompts, training LLMs, fine-tuning image-to-text models like BLIP, or extracting insights from the data—like the most common combinations of modifiers.
2022 · github.com
- 13
- 14EO
I built this as a personal open-source project to explore how EU AI Act requirements can be translated into concrete, inspectable technical checks. The core idea is local-first compliance: – risk classification (Articles 5–15, incl. prohibited use cases) – bias evaluation using CrowS-Pairs – automatic Annex IV–oriented PDF reports – no cloud services or external APIs (browser-based + Ollama) I’m especially interested in feedback on whether this kind of technical framing of AI regulation makes sense in real-world projects.
Jan 2026 · github.com
- 15
Real-time observability dashboard for OpenClaw AI agents
Feb 2026 · clawmetry.com
- 16
- 17OS
Hi HN, Hugh and Vince here. LLMonitor helps you record, trace & search your LLM queries and chatbot conversations. You can also capture user feedback on your frontend and correlate it with backend LLM queries then use that to fine-tune your own models. The project started has an internal tool in our previous (failed) AI startup. We’re aware the LLM observability space is very crowded. Apart from being open-source, we differentiate with: - Model-agnostic and minimal lock-in (no MITM of requests). - High focus on DX and dashboard clarity. - Support for complex scenarios: e.g. a chatbot that…
2023 · github.com
- 18
- 19PE
Nowadays, a common AI tech stack has hundreds of different prompts running across different LLMs. Three key problems: - Choices, picking from 100s of LLMs the best LLM for that 1 prompt is gonna be challenging, you're probably not picking the most optimized LLM for a prompt you wrote. - Scaling/Upgrading, similar to choices but you want to keep consistency of your output even when models depreciate or configurations change. - Prompt management is scary, if something works, you'll never want to touch it but you should be able to without fear of everything breaking. So we launched Prompt…
2024 · jigsawstack.com
- 20OS
EU legislation (which affects UK and US companies in many cases) requires being able to truly reconstruct agentic events. I've worked in a number of regulated industries off & on for years, and recently hit this gap. We already had strong observability, but if someone asked me to prove exactly what happened for a specific AI decision X months ago (and demonstrate that the log trail had not been altered), I could not. The EU AI Act has already entered force, and its Article 12 kicks-in in August this year, requiring automatic event recording and six-month retention for high-risk systems,…
Mar 2026
- 21AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 22OO
Hey HN, we're super excited to share something we've been working on: OpenLIT. After an engaging preview that some of you might recall, we are now proudly announcing our first stable release! *What's OpenLIT?* Simply put, OpenLIT is an open-source tool designed to make monitoring your Large Language Model (LLM) applications straightforward. It’s built on OpenTelemetry, aiming to reduce the complexities that come with observing the behavior and usage of your LLM stack. *Beyond Basic Text Generation:* OpenLIT isn’t restricted to just text and chatbot outputs. It now includes automatic…
2024 · github.com
- 23LS
Hi, I was a corporate lawyer for many years working with a lot of financial services and insurance companies. In practicing law, I noticed there was a lot of repetition in the tasks I was working on even as a highly paid attorney that could be automated. I wanted to solve the problem of dealing with a lot information and data in a practical way, using AI. This motivated me to start AI Bloks/LLMWare with my husband, who had a deep background in software and is a very early adopter of AI. We have been on this journey with our open source project LLMWare for the past 4 months, producing a…
2024 · github.com
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →