LLM Inference Calculator – Estimate VRAM, Latency, and Throughput
LLM Inference Calculator — Estimate throughput, latency, TTFT, TPOT, and GPU memory usage for large language model inference. LLM 推理计算器 — 估算大模型推理吞吐量 (throughput)、延迟 (latency)、TTFT、TPOT 与 GPU 显存占用。
LLM Inference Calculator – Estimate VRAM, Latency, and Throughput launched on August 28, 2026 with 6 votes, #1,590 of 23,274 launches that month and more than 75% of that year's launches.
Does a similar job
all alternatives →
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
- IMI made a GPU VRAM calculator for transformer-based models2023 · vram.asmirnov.xyz · ▲135
SelfHostLLM2025 · selfhostllm.org · ▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
Find the best local LLM for your hardware, ranked by benchmarksMay 2026 · github.com · ▲283Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm
LLM StatsOct 2025 · llm-stats.com · ▲293I was overspending on GPT-4o. It was really hard to compare different models I could switch to, so I built this LLM comparison tool. It shows leaderboards, pricing, and performance data across 100+ LLMs (including all major providers and open-source models). Key features: - Live pricing comparisons - Benchmark Scores (MMLU, HumanEval, GPQA, etc.) - Context length vs cost analysis - Speed/throughput tests across providers - Quality vs price visualizations - Open source (all data verifiable) Try it out: https://llmstats.com I'd like to know your opinion :) Tech stack: Next.js,…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗
Launched alongside, August 2026
the whole month →- TL
Life & fun · Aug 2026 · louisabraham.github.io
Clipto MCP▲655Hey HN, We recently built Clipto. It’s a tool that lets you search over terabytes of video, audio, and images on your computer, without relying on the cloud. Motivation: we probably all had this similar experience, we know a moment exists in a video or audio, but finding it takes hours scrubbing the timeline. You can send all the media to process in the cloud, but it’s slow, expensive and raises privacy concerns. So we decided to build our own on-device media search engine. How it works (high level): 1. We ingest video, audio, image; normalize formats via ffmpeg; run content analysis to…
Work · Aug 2026 · clipto.com
Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…
Life & fun · Aug 2026 · toneyalexander.github.io
Astute▲606Automate your B2B brand going viral, with new media creators
AI · Aug 2026 · company-app.joinastute.com