AI · alternatives · 2026
24 alternatives to Inferencer – Run and deeply control local AI models (macOS release)
Private inference app that lets you see the token entropy, explore and change the token probabilities. Just released on macOS, iOS version next then other platforms. Here's a demo of it in action running DeepSeek Terminus: https://youtu.be/kts098EL2PQ Would love to hear any feedback or feature requests from the community.
Inferencer is a macOS application that runs AI models locally while giving users deep visibility into how the models generate responses. It displays token entropy and allows users to explore and modify token… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com · its alternatives →
- 2
Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
Jul 2026 · github.com · its alternatives →
- 3
- 4
- 5
- 6
Deploying vision models is time consuming and tedious. Setting up dependencies. Fixing conflicts. Configuring TRT acceleration. Flashing (and re-flashing) NVIDIA Jetsons. A streamlined, developer-friendly solution for inference is needed. We, the Roboflow team, have been hard at work open sourcing Inference, an open source vision deployment solution. Our solution is designed with developers in mind, offering a HTTP-based interface. Run models on your hardware without having to write architecture-specific inference code. Here's a demo showing how to go from a model to GPU inference on a video…
2023 · github.com · its alternatives →
- 7
Fast multimodal-native inference at scale
Dec 2025 · gmicloud.ai · its alternatives →
- 8
- 9
General Compute▲315AI models that run on an inference cloud optimized for speed
May 2026 · generalcompute.com · its alternatives →
- 10
Osaurus▲534Open source agents that run 100% locally on your Mac
Jul 2026 · osaurus.ai · its alternatives →
- 11
Opper AI▲229Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use. You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model. You can also run a debate round where models see each other's reasoning and get a…
Jul 2026 · opper.ai · its alternatives →
- 12
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 13
TLDR: A small, vendor-agnostic inference loop that turns token logprobs/perplexity/entropy into an extra pass and reasoning for LLMs. - Captures logprobs/top-k during generation, computes perplexity and token-level entropy. - Triggers at most one refine when simple thresholds fire; passes a compact “uncertainty report” (uncertain tokens + top-k alts + local context) back to the model. - In our tests on technical Q&A / math / code, a small model recovered much of “reasoning” quality at ~⅓ the cost while refining ~⅓ of outputs. I kept seeing “reasoning” models behave…
2025 · github.com · its alternatives →
- 14
Local▲107Super excited to launch our new app Local today. What we’ve learned at Base Compute over the last months is that running AI directly on your laptop or workstation gives you maximum privacy and it’s free, but it’s also a massive headache to configure. So we’ve decided what matters is making the experience completely frictionless for users. Local analyses the hardware of your laptop, optimises the AI for it, and recommends the best models for your specific device. It let’s you do what you’re doing with cloud AI already, just for free and on your own machine: Chatting with PDF’s, Recording and…
Aug 2026 · basecompute.co · its alternatives →
- 15IB
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…
2025 · tinytpu.com · its alternatives →
- 16
A uniform interface to run deep learning models from multiple frameworks - uber/neuropod
2020 · github.com · its alternatives →
- 17BA
2019 · brancher.org · its alternatives →
- 18
Hi HN, I built OpenGraviton, an open-source AI inference engine designed to push the limits of running extremely large models on consumer hardware. The system combines several techniques to drastically reduce memory and compute requirements: • 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic sparsity with Top-K pruning and MoE routing • mmap-based layer streaming to load weights directly from NVMe SSDs • speculative decoding to improve generation throughput These allow models far larger than system RAM to run locally. In early benchmarks, OpenGraviton reduced…
Mar 2026 · opengraviton.github.io · its alternatives →
- 19
Hi HN, I'm Paul from Tensordyne. We build AI inference systems and chips on logarithmic math. We've put together an interactive Token Economics Calculator to help make apples-to-apples comparisons of inference hardware across vendors: We're interested in how closely it lines up with the community's view of the market. Why we built this Investors and customers kept asking how our system compares to others (NVIDIA and a growing list of startups). Plenty of publicly available data exists, but it's scattered and inconsistent. News articles, provider sites, Artificial Analysis, MLCommons, and now…
Nov 2025 · tensordyne.ai · its alternatives →
- 20GA
2021 · inferrd.com · its alternatives →
- 21
Most inference UIs that I've come across pretty much just give us a chat-like interface to toy around with models in a single visual conversation thread. Given the fact that we are limited to seeing only one output at a time, it's kind of hard to compare outputs from different models, adjustments made to the prompting, and sampler settings. But even when keeping the generation parameters the same (e.g., to test for reliability in the output) and just going for multiple passes, there is no easy way to have a side-by-side comparison to keep track of the outputs from the multiple "rounds". I…
2024 · github.com · its alternatives →
- 22
Lekh AI▲80Run powerful AI models on your Mac:fully offline and private
Jan 2026 · apps.apple.com · its alternatives →
- 23DS
2020 · inferrd.com · its alternatives →
- 24
Inference hosting for AI teams who ship fast and scale faster.
2022 · banana.dev · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →