Alternatives
Products that do what AKM does
Rate-limit aware smart API key scheduling
- 1
- 2
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 3
- 4
- 5OA
A lightweight engine for durable execution / deterministic workflows I built with Rust, wasmtime and the WASM Component Model. Its main use is running reliable, long-running workflows that can automatically resume after failures. Looking for feedback on the approach and potential use cases!
2025 · obeli.sk
- 6
- 7KP
2015 · keymetrics.io
- 8
- 9
- 10FF
2019 · github.com
- 11UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024
- 12
- 13DR
Hi HN! I built a distributed rate limiter using the token bucket algorithm with Redis backing. Key highlights: • 50,000+ requests/second throughput with <2ms P95 latency • Redis-backed distributed state for multi-instance deployments • 18 REST API endpoints for rate limiting, config, and monitoring • 265+ tests including load tests and integration tests • Docker/Kubernetes ready with comprehensive documentation Built with Java 21 + Spring Boot. Perfect for protecting APIs, microservices, or SaaS platforms from abuse. The pain point I solved: existing solutions were either too…
Sep 2025 · github.com
- 14
- 15BA
2014 · github.com
- 16
- 17AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 18GR
2018 · github.com
- 19RC
2016 · github.com
- 20NB
2022 · pypi.org
- 21KD
I built this after seeing multiple teams accidentally ship API keys in their frontend code. The problem: Modern web development moves fast. You're vibe-coding, shipping features, and suddenly your AWS keys are sitting in a tag visible to anyone who opens DevTools. I've personally witnessed this happen to at least 3-4 production apps in the past year alone. KeyLeak Detector runs through your site (headless browser + network interception) and checks for 50+ types of leaked secrets: AWS/Google keys, Stripe tokens, database connection strings, LLM API keys (OpenAI, Claude, etc.), JWT…
Nov 2025 · github.com
- 22KC
I think in-process key management is the right abstraction for multi-key LLM setups. Not LiteLLM, not a Redis queue, not a custom load balancer. The failure modes are well-understood: a key gets rate-limited, you wait, you try the next one. Billing errors need a longer cooldown than rate limits. This is not a distributed systems problem — it's a state machine that fits in a library. The problem is everyone keeps solving it with infrastructure instead. Spin up LiteLLM, now you have a Python service to maintain. Reach for Redis, now you have a database for a problem that doesn't need one.…
Mar 2026 · github.com
- 23AC
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Apr 2026
- 24WB
Over the past few months, as we scaled our internal AI Agents, we hit a dead end: Running LLM-generated arbitrary code in Docker is basically running naked on security due to container escape risks. But using full traditional VMs takes minutes to boot and eats too much memory to support high-density concurrency. We loved the developer experience of SaaS sandboxes on the market, but they are closed-source, expensive, and have too high a barrier to entry for self-hosting. So, our team decided to build our own. After months of grinding, using RustVMM and KVM, we built a blazing-fast,…
Apr 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →