I built a tool showing how AI providers (should) throttle their models
OP here: this project was born out of the frustration/paranoia that AI providers are throttling their models when their server load is too high. So, I set out to model and study the problem mathematically to understand what was happening, what I found was quite surprising. The idea seems natural: as the data center demand increases momentarily through the day, throttling their models (either using quantized versions, reducing the context window or lowering the tier of the model to a smaller one) seems appealing as the replacement model in principle uses less electricity. The problem is…
I built a tool showing how AI providers (should) throttle their models launched on August 28, 2026 with 6 votes, #1,590 of 23,274 launches that month and more than 75% of that year's launches.
What it does
When a demand storm hits an LLM fleet, one routing rule decides who still gets the strong model. A playable figure from
In the maker’s words, at launch
OP here: this project was born out of the frustration/paranoia that AI providers are throttling their models when their server load is too high. So, I set out to model and study the problem mathematically to understand what was happening, what I found was quite surprising. The idea seems natural: as the data…
Does a similar job
all alternatives →
Opper AIJul 2026 · opper.ai · ▲229Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use. You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model. You can also run a debate round where models see each other's reasoning and get a…
I ran a language model on a PS2Mar 2026 · github.com · ▲46The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
- RLRate limiting, caching and request prioritization for AI apps2024 · fluxninja.com · ▲10
Generative AI applications pose a unique challenge in production. They are computationally intensive and orders of magnitude slower than traditional data-intensive applications. Scaling these applications is further complicated by expensive hardware requirements and GPU shortages. Consequently, developers are scrambling to implement home-grown caching and rate-limiting solutions, which are error-prone and difficult to get right. FluxNinja Aperture delivers a production-grade experience with a purpose-built load management platform that provides rate & concurrency limiting, caching, and…
Serve 100 Large AI models on a single GPU with low impact to TTFTNov 2025 · github.com · ▲7I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
- TCThe Cascade Graph – An interactive map of AI and energy constraintsJun 2026 · atomprophet.io · ▲26
Hello, I wanted to share with you all a interactive map of the economics and physics constraints of the AI buildout. It has macro drivers, industrial chokepoints, and where that shows up in markets. I've added 393 nodes and 562 edges to capture other supply / physics constraints as well. There's no sign up, and no pay wall, it's all free. Please let me know what you think!
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, August 2026
the whole month →- TL
Life & fun · Aug 2026 · louisabraham.github.io
Clipto MCP▲655Hey HN, We recently built Clipto. It’s a tool that lets you search over terabytes of video, audio, and images on your computer, without relying on the cloud. Motivation: we probably all had this similar experience, we know a moment exists in a video or audio, but finding it takes hours scrubbing the timeline. You can send all the media to process in the cloud, but it’s slow, expensive and raises privacy concerns. So we decided to build our own on-device media search engine. How it works (high level): 1. We ingest video, audio, image; normalize formats via ffmpeg; run content analysis to…
Work · Aug 2026 · clipto.com
Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…
Life & fun · Aug 2026 · toneyalexander.github.io
Astute▲606Automate your B2B brand going viral, with new media creators
AI · Aug 2026 · company-app.joinastute.com