SearchAI Inference Server
Run Private LLMs on CPUs.
What it does
SearchAI Inference Server runs private AI models inside your network and serves them through one OpenAI-compatible endpoint — chat, RAG, function calling, JSON output, vision, video, speech, and image editing. No data egress. No metered billing. No GPUs required. Every install ships with a built-in console of 380 tested prompts — document understanding, extraction to JSON, function calling, classification, multilingual, vision, video, and speech scenarios, purpose-built for enterprises.
Download the SearchAI Inference Server: private LLMs with an OpenAI-compatible API — chat, RAG, function calling, vision, video, and speech, entirely on your own infrastructure with fixed-cost scaling.
SearchAI Inference Server runs private AI models inside your network and serves them through one OpenAI-compatible endpoint — chat, RAG, function calling, JSON output, vision, video, speech, and image editing. No data egress. No metered billing. No GPUs required. A Linux host (Ubuntu 22.04+, Debian 12+, Amazon Linux 2023, RHEL 9+ — x86_64 with AVX2, or arm64), an Apple Silicon Mac (macOS 13+), 64-bit Windows, or Docker — with outbound HTTPS. Details in the supported-platforms table . On Windows Server or desktop? A native Windows x64 build installs with one PowerShell line — it takes a few extra steps (elevated PowerShell, SmartScreen on the unsigned binary), so follow the step-by-step…from inference-server.searchblox.com
Does the same job
all alternatives →
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, September 2026
the whole month →
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API. - carloslfu/slotstream
Dev tools · 4d ago · github.com
NPC-Forge is a framework for building conversational agents that run on the CPU without relying on machine learning or LLMs. - NPC-Forge/docs/development.md at main · gioblu/NPC-Forge
AI · 2d ago · github.com
Weedout is a Safari extension that filters out videos YouTube labels as “Made with AI”.
AI · 4d ago · masteranza.github.io
Find your next hire or your next role from Hacker News monthly threads. AI-powered matching between candidates and job postings.
AI · 4d ago · hnmatchmaker.com
I made OwnTime to balance between competing priorities in my life. The idea is based on two main influences: the concept of "roles" from "The 5 Choices" (2015), and my repeated failure to effectively implement time blocking due to the necessary flexibility in my role. The app allows you to define time budgets, which are not much more than a few mutually exclusive countdown timers. The UX is essentially that of a chess clock for an arbitrary number of players with configurable time. The whole point of the app is to kick you out of the running role/priority when its time is up. One…
Life & fun · 4d ago · owntime.app