Alternatives
Products that do what LLMs consume 5.4x less mobile energy than ad-supported web search does
The standard AI energy debate compares server-side LLM inference to a server-side Google query. I think this misses most of what actually happens on a mobile device during a real search session. I built a parametric model of the full end-to-end mobile search session: 4G/5G radio energy, SoC rendering cost for a 2.5MB page, programmatic advertising RTB auctions running in the background, and network transmission costs for both sides. Then compared it to an equivalent LLM session. Main finding across 10,000 Monte Carlo draws: on mobile, a standard LLM session uses on average 5.4x less…
- 1WW
I spent a few hours last weekend testing whether AI can replace code by executing directly. Built a contact manager where every HTTP request goes to an LLM with three tools: database (SQLite), webResponse (HTML/JSON/JS), and updateMemory (feedback). No routes, no controllers, no business logic. The AI designs schemas on first request, generates UIs from paths alone, and evolves based on natural language feedback. It works—forms submit, data persists, APIs return JSON—but it's catastrophically slow (30-60s per request), absurdly expensive ($0.05/request), and has zero UI…
Nov 2025 · github.com
- 2IV
The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
2024 · github.com
- 3
- 4
- 5RC
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
2024 · edgematic.dev
- 6AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 7AL
Try it out here: https://labs.refuel.ai/playground Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction.
2023
- 8OS
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com
- 9IB
I was overspending on GPT-4o. It was really hard to compare different models I could switch to, so I built this LLM comparison tool. It shows leaderboards, pricing, and performance data across 100+ LLMs (including all major providers and open-source models). Key features: - Live pricing comparisons - Benchmark Scores (MMLU, HumanEval, GPQA, etc.) - Context length vs cost analysis - Speed/throughput tests across providers - Quality vs price visualizations - Open source (all data verifiable) Try it out: https://llmstats.com I'd like to know your opinion :) Tech stack: Next.js,…
2025 · llm-stats.com
- 10NL
Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction. Here is the interactive demo: https://labs.refuel.ai/playground. Pretty fun to play with!
2023
- 11AU
Hi HN, I was once given the advice: Don't waste expensive frontier model credits (GPT/Claude/etc.) on bulk work. Send the boring, repetitive, high-volume jobs to a smaller model, and save the expensive prompts for when you actually need frontier-level reasoning. I complained and told my manager that I shouldnt have to think about using certain models for certain coding tasks, and that one model should handle everything. Well, here we are anyway. If anyone needs a place to absolutely abuse an LLM with high-volume tasks, come beat ours up at https://yolo-auto.com. Here are…
Jul 2026 · yolo-auto.com
- 12BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
Oct 2025 · docs.butter.dev
- 13IB
I’ve been playing around with local LLMs for the past couple of months and decided to build something that can run on an iPhone. It’s a universal app built with SwiftUI and the excellent ggml library. The model is an SFT fine tuned and 4 bit quantised version of the RedPajama-INCITE-Chat-3B-v1 OSS LLM. It works reasonably well on recent-ish (~3 year old) iPhones, iPads and Macs. It was launched on the App Store yesterday[1] and Product Hunt today[2]. It seems to be reasonably ok at natural language interactions, but given its size, does pretty badly at coding and reasoning. Also, it…
2023
- 14
- 15TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024
- 16IO
Hey folks, I’m the creator of WFGY — a semantic reasoning framework for LLMs. After open-sourcing it, I did a full technical and value audit — and realized this engine might be worth $8M–$17M based on AI module licensing norms. If embedded as part of a platform core, the valuation could exceed $30M. Too late to pull it back. So here it is — fully free, open-sourced under MIT. --- ### What does it solve? Current LLMs (even GPT-4+) lack *self-consistent reasoning*. They struggle with: - Fragmented logic across turns - No internal loopback or self-calibration - No modular thought units - Weak…
2025 · github.com
- 17LS
Hi HN, I built llm.sql, an LLM inference framework that reimagines the LLM execution pipeline as a series of structured SQL queries atop SQLite. The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth). When available memory is less than the model size and KV cache, the OS incurs page faults and swaps pages using LRU-like strategies, resulting in throughput degradation that's hard to notice and even harder to debug. In fact, the memory access pattern during LLM inference is deterministic - we know exactly which weights are needed…
Apr 2026
- 18IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 19
- 20RE
Recent academic work ([1], [2]) has suggested that LLMs can effectively simulate different Internet subpopulations. For example, you may ask ChatGPT to emulate being a high school teacher explaining Newton’s laws of physics. Building upon this, we created Roundtable, a platform that uses LLMs to predict how people will respond to any arbitrary survey question. To do so, we needed to first reduce bias arising from GPT’s training procedure. Because these models are primarily trained on Internet data, they can be heavily skewed towards the demographics of heavy Internet users (e.g.,…
2023 · roundtable.ai
- 21GV
Hey HN, I just updated my project that compares some LLMs. It uses your prompt for all the models and runs at the same time. You can see the results being generated in real-time and decide what's the best for your use case. I'm open to any suggestions and feedback. Thanks!
2024 · geminivsgpt.com
- 22IB
Hey HN, I've been working on something cool that I wanted to share with you all. It's called Viewpoint, an analytics tool for LLMs like OpenAI, Anthropic models, and Gemini. The idea came from the constant flood of new LLM models and the need to figure out which ones work best for my projects without breaking the bank. With viewpoint, I can track token usage, costs, latency(WIP), and traffic over time, making it easier to compare different models and see which ones perform best and save money. The tool works asynchronously, so it doesn't add any latency to your LLM requests, and you have…
2024 · viewpointhq.com
- 23AO
Hi, We are building an open-source framework for loading and structuring LLM context to create accurate and explainable LLM answers using knowledge graphs and vector stores. We built the tool with four main concepts in mind: 1. Loader -> uses dlt in the backend to load and structure the data 2. Cognify step -> creates a graph with summaries, labels and factoids that are interconnected across the documents and stored as a representation in the vector store 3. Optimizer -> Uses DSPy to optimize LLM queries, and we plan to extend it to most of the knobs we can turn, like chunking etc. 4. Search…
2024 · github.com
- 24NT
With the latest launch from Google I've added support for Gemma 3 270M, the speed for local LLM to TTS token time is incredible! This is an heavy obvious work in progress - any contributions or tips would be welcome. The idea is to have a fast moving edge model playground, and maybe have some utility (like the e reader) on the side.
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →