Alternatives
Products that do what TokenScope does
Real-time, privacy-first monitoring of your LLM API usage
- 1
- 2AM
I built this out of curiosity about what Claude Code was actually sending to the API. Turns out, watching your tokens tick up in real-time is oddly satisfying. Sherlock sits between your LLM tools and the API, showing you every request with a live dashboard, and auto-saved copies of every prompt as markdown and json.
Jan 2026 · github.com
- 3TA
TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.). It’s written in C++ 17 with thin Python bindings, keeps the exact same BPE vocab/special-token rules, and focuses on raw speed. I’m teaching myself LLM internals by re-implementing the stack from first principles. Profiling TikToken’s Python/Rust implementation showed a lot of time was spent doing regex matching. Most of my perf gains come from a) using a faster jit-compiled regex engine; and b) simplifying the algorithm to forego regex matching special tokens at all.…
2025 · github.com
- 4
- 5
- 6LT
2023 · github.com
- 7YD
If you've built any web-based app in the last 15 years, you probably used something like Datadog, New Relic, Sentry, etc. to monitor and trace your app, right? Why should it be different when the app you're building happens to be using LLMs? So today we're open-sourcing OpenLLMetry-JS. It's an open protocol and SDK, based on OpenTelemetry, that provides traces and metrics for LLM JS/TS applications and can be connected to any of the 15+ tools that already support OpenTelemetry. Here's the repo: https://github.com/traceloop/openllmetry-js A few months ago we launched…
2024 · github.com
- 8AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 9
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 10
- 11
- 12
- 13
- 14LL
Author here. I wanted to keep my conversation with #Gemini about code handy while discussing something creative with #ChatGPT and using #DeepSeek in another window. I think it's a waste to have Electron apps and so wanted to chat with LLMs on my own terms. When I discovered the llm CLI tool I really wanted to have convenient and pretty looking access to my conversations, and so I wrote gtk-llm-chat - a plugin for llm that provides an applet and a simple window to interact with LLM models. Make sure you've configure llm first (https://llm.datasette.io/en/stable/) I'd…
2025 · github.com
- 15
- 16SW
Chrome now includes a native on-device LLM (Gemini Nano) starting in version 138. I've been building with it since it was in origin trials, it's powerful but the official Prompt API is still a bit awkward: - Enforces sessions even for basic usage - Requires user-triggered downloads - Lacks type safety or structured error handling So I open-sourced a small TypeScript wrapper I originally built for other projects to smooth over the rough edges: github: https://github.com/kstonekuan/simple-chromium-ai npm: https://www.npmjs.com/package/simple-chromium-ai…
2025 · github.com
- 17
- 18
See exactly what your Claude and ChatGPT conversations cost
Jun 2026 · chromewebstore.google.com
- 19
- 20
- 21TO
I built TraceAIO, an open-source tool that prompts LLMs on your behalf and tells you whether ChatGPT, Perplexity, and Gemini mention your brand — and which competitors and sources show up instead. Yeah, this category smells a bit like a grift, same as early SEO. And I think over time it will become just SEO again, and become about good content. The tool just helps you monitor over time. It queries the browser products through real browser sessions, not APIs, runs on Docker, with an MCP server so you can query your own data through an LLM. No business model, Apache 2.0, self hosted. If you…
Jun 2026 · traceaio.org
- 22TA
2024 · franz101.substack.com
- 23
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →