Inferencer – Run and deeply control local AI models (macOS release)
Private inference app that lets you see the token entropy, explore and change the token probabilities. Just released on macOS, iOS version next then other platforms. Here's a demo of it in action running DeepSeek Terminus: https://youtu.be/kts098EL2PQ Would love to hear any feedback or feature requests from the community.
Inferencer – Run and deeply control local AI models (macOS release) launched on September 24, 2025 with 15 votes, #703 of 8,160 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
Inferencer is a macOS application that runs AI models locally while giving users deep visibility into how the models generate responses. It displays token entropy and allows users to explore and modify token probabilities in real time, providing fine-grained control over model inference. The app is designed for users who want privacy and transparency when working with AI models, with an iOS version and additional platform support planned.
written from the facts on this page · September 2026
From the sources
Inferencer lets you download, run, and fully control the latest SOTA AI models locally. Advanced inferencing and parental controls let you fine‑tune accuracy, inspect token probabilities in real time, and use prompt framing to guardrail the output generated.
Run AI models locally with full control and privacy. Inspect token probabilities, serve models over network, stream large models with memory offloading and more.
By default, all AI processing happens offline and on your device. No data is sent to the cloud for processing. ⓘ If enabled, the server feature allows you to serve and connect to your own or trusted devices. No data is sent elsewhere. State-of-the-Art Support for SOTA models. Fastest inferencing performance. Patent Pending deep learning inferencing to fully control your models. Token Inspection Tap on the token inspector icon to reveal the probabilities for each token, showing you exactly what the model was thinking at each step. Token Entropy : Instantly see contentious tokens, allowing you to better gaguge the confidence of the generation. Token Selection : Specifically select tokens to…from inferencer.com
Does a similar job
all alternatives →
We made our own inference engine for Apple Silicon2025 · github.com · ▲186We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series MacJul 2026 · github.com · ▲919Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
Pip install inference, open source computer vision deployment2023 · github.com · ▲71Deploying vision models is time consuming and tedious. Setting up dependencies. Fixing conflicts. Configuring TRT acceleration. Flashing (and re-flashing) NVIDIA Jetsons. A streamlined, developer-friendly solution for inference is needed. We, the Roboflow team, have been hard at work open sourcing Inference, an open source vision deployment solution. Our solution is designed with developers in mind, offering a HTTP-based interface. Run models on your hardware without having to write architecture-specific inference code. Here's a demo showing how to go from a model to GPU inference on a video…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
Launched alongside, September 2025
the whole month →
The site hit #1 on HN about a year ago. Got enough traffic that I turned it into a proper API. What's there: - REST API — 35k products, manually filterable by category, free key, 60 req/min - CLI — npx anycrap random -c food - faker.js plugin — bundled, no API call, works offline - HuggingFace dataset — full 35k rows for ML/training https://anycrap.shop/developers
AI · 2025 · anycrap.shop
I made a built-from scratch Wayland Compositor to display any GUI app* in the terminal! I think there is a lot of unexplored potential in custom Wayland compositors, a lot of really cool things you can embed existing applications into! So, I started with embedding apps into the terminal because that is the easiest input/output (output is just utf-8 and I use the great `chafa` library for that, and I just read from stdin for the input). If you have any other ideas for cool Wayland compositors, let me know. I purposedly wrote 80% the app in Typescript to appeal to the most developers and…
Dev tools · 2025 · github.com
- IR
Years ago I stumbled across a basic version of this concept and it stuck with me. I knew if I was ever going to take on such a project, it would need to be flawless, but without coding experience it was just another idea that would never happen. By the end of 2024, as AI coding tools exploded everywhere, I finally had a way to make it real. I started from zero knowledge and spent months collaborating with AI agents as a learning experience. Every pixel and every function went through me. The AI translated what I asked for into code, but every decision was human. I didn't use existing OS…
AI · 2025 · mitchivin.com
