Lumina – Open-source observability for LLM applications
Hey HN! I built Lumina – an open-source observability platform for AI/LLM applications. Self-host it in 5 minutes with Docker Compose, all features included. The Problem: I've been building LLM apps for the past year, and I kept running into the same issues: - LLM responses would randomly change after prompt tweaks, breaking things - Costs would spike unexpectedly (turns out a bug was hitting GPT-4 instead of 3.5) - No easy way to compare "before vs after" when testing prompt changes - Existing tools were either too expensive or missing features in free tiers What I Built: Lumina is…
What it does
In the maker’s words, at launch
Hey HN! I built Lumina – an open-source observability platform for AI/LLM applications. Self-host it in 5 minutes with Docker Compose, all features included. The Problem: I've been building LLM apps for the past year, and I kept running into the same issues: - LLM responses would randomly change after prompt tweaks, breaking things - Costs would spike unexpectedly (turns out a bug was hitting GPT-4 instead of 3.5) - No easy way to compare "before vs after" when testing prompt changes - Existing tools were either too expensive or missing features in free tiers What I Built: Lumina is OpenTelemetry-native, meaning: - Works with your existing OTEL stack (Datadog, Grafana, etc.) - No vendor lock-in – standard trace format - Integrates in 3 lines of code Key features: - Cost & quality monitoring – Automatic alerts when costs spike or responses degrade - Replay testing – Capture production traces, replay them after changes, see diffs - Semantic comparison – Not just string matching – uses Claude to judge if responses are "better" or "worse" - Self-hosted tier – 50k traces/day, 7-day retention, ALL features included (alerts, replay, semantic scoring) How it works: Start Lumina git clone https://github.com/use-lumina/Lumina cd Lumina/infra/docker docker-compose up -d // Add to your app (no API key needed for self-hosted!) import { Lumina } from '@uselumina/sdk'; const lumina = new Lumina({ endpoint: 'http://localhost:8080/v1/traces', }); // Wrap your LLM call const response = await lumina.traceLLM( async () => await openai.chat.completions.create({...}), { provider: 'openai', model: 'gpt-4', prompt: '...' } ); That's it. Every LLM call is now tracked with cost, latency, tokens, and quality scores. What makes it different: 1. Free self-hosted with limits that work – 50k traces/day and 7-day retention (resets daily at midnight UTC). All features included: alerts, replay testing, semantic scoring. Perfect for most development and small production workloads. Need more? Upgrade to managed cloud. 2. OpenTelemetry-native – Not another proprietary format. Use standard OTEL exporters, works with existing infra. Can send traces to both Lumina AND Datadog simultaneously. 3. Replay testing – The killer feature. Capture 100 production traces, change your prompt, replay them all, get a semantic diff report. Like snapshot testing for LLMs. 4. Fast – Built with Bun, Postgres, Redis, NATS. Sub-500ms from trace to alert. Handles 10k+ traces/min on a single machine. What I'm looking for: - Feedback on the approach (is OTEL the right foundation?) - Bug reports (tested on Mac/Linux/WSL2, but I'm sure there are issues) - Ideas for what features matter most (alerts? replay? cost tracking?) - Help with the semantic scorer (currently uses Claude, want to make it pluggable) Why open source: I want this to be the standard for LLM observability. That only works if it's: - Free to use and modify (Apache 2.0) - Easy to self-host (Docker Compose, no cloud dependencies) - Open to contributions (good first issues tagged) The business model is managed hosting for teams who don't want to run infrastructure. But the core product is and always will be free. Try it: - GitHub: https://github.com/use-lumina/Lumina - Demo video: [YouTube link] - Docs: https://docs.uselumina.io - Quick start: 5 minutes from `git clone` to dashboard I'd love to hear what you think! Especially interested in: - What observability problems you're hitting with LLMs - Missing features that would make this useful for you - Any similar tools you're using (and what they do better) Thanks for reading!
Does the same job
all alternatives →
OpenLIT's Zero-code LLM ObservabilityOct 2025 · ▲136Trace LLM requests + costs with OpenTelemetry monitoring
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
