Ferrum

What it does

Ferrum is an open-source, Rust-native runtime for running and serving local language models on Apple Silicon Metal and NVIDIA CUDA. One binary gives you an interactive CLI plus OpenAI-compatible Chat Completions and Responses APIs, without requiring Python, PyTorch, or vLLM at runtime. Choose a model explicitly, run it locally, or expose the same model over HTTP.

Local LLM inference and an OpenAI-compatible server in one Rust binary. Ferrum 0.14.0 unifies the execution architecture.

Terminal chat and an OpenAI-compatible API, without a Python runtime. Goal: maximize output throughput while meeting TTFT, TPOT and ITL SLOs. Fix hardware, model and output policy; choose execution policy π from eligible policies Π under predeclared SLOs. Installers verify checksums and configure PATH. Open a new terminal on macOS/Linux; PowerShell is ready immediately. Repeat the command to upgrade; models and configuration are preserved. macOS uses Metal. Linux/WSL selects sm89 (CUDA 12.4) or sm120 (CUDA 12.8) when all visible GPUs report the matching compute capability. Compatible NVIDIA driver and CUDA/NCCL runtimes are required. Windows and Homebrew CUDA target sm89; Windows bundles…from ferrum.pandaailabs.com

• 
  Ferrum.js – Rust features in JS: traits and advanced iteration2019 · github.com · ▲63

  Features from the rust language in javascript: Provides Traits/Type classes & a hashing infrastructure and an advanced library for working with sequences/iterators in js - adobe/ferrum

• 
  ChattyUI2024 · ▲149

  Run open-source LLMs locally in the browser using WebGPU

• 
  Ollama – Run LLMs on your Mac2023 · github.com · ▲284

  Hi HN A few folks and I have been working on this project for a couple weeks now. After previously working on the Docker project for a number of years (both on the container runtime and image registry side), the recent rise in open source language models made us think something similar needed to exist for large language models too. While not exactly the same as running linux containers, running LLMs shares quite a few of the same challenges. There are "base layers" (e.g. models like Llama 2), specific configuration to run correctly (parameters, temperature, context window sizes etc). There's…

• WM
  We made glhf.chat – run almost any open-source LLM, including 405B2024 · glhf.chat · ▲161

  Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…

• 
  LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)2023 · github.com · ▲45

  G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…

• 
  npm i -g hotcellAug 2026 · github.com · ▲97

  Hi HN! Sina here. i wanted to share this open source project (apache 2.0) that i've been working on for the past month or so. it's called hotcell and it lets you create/pause/manage sandboxes on any device (your laptop, linux vm, bare metal). i've tested it against various benchmarks that i found on computesdk (https://www.computesdk.com/benchmarks/sandboxes/) as well as the one shared by dax (cofounder of open code) on twitter recently (link to his original thread and my reply https://x.com/sinasanm/status/2078158598996443346) and…

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 18d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 18d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 20d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 21d ago · stillwet.art

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 30d ago · world.pixverse.video

• 
  Beam.win▲378

  Run AI agents on one Mac and use its windows on another

  AI · 5d ago · beam.win

• 
  An e-ink frame that hears birds and draws them as 1800s illustrations▲2,389

  E-ink bird frame for Raspberry Pi - real-time bird detection by audio, fully local AI, rendered as real, hand-cut 1800s bird illustrations. - arnegiacomo/fugleramme

  AI · Sep 2026 · github.com

• WI
  What if the speed of light was 5 km/h?▲629

  Special relativity simulator in which light crawls along at a human speed, so Lorentz contraction, aberration and Doppler shift happen at walking pace.

  Life & fun · Sep 2026 · rivendell.dmitrybrant.com

• 
  Jev▲544

  Fast, structured AI decisions for software automation

  AI · Sep 2026 · console.typesafe.ai

• 
  ReSO AI▲510

  Get discovered & recommended when customers ask AI

  AI · Sep 2026 · resollm.ai

• 
  Kilo Code for iOS and Android▲498

  Start coding agents, control sessions, review PRs. Anywhere.

  Dev tools · Sep 2026 · kilo.ai

• 
  AINA▲497

  Find your job search blind spots with an AI coach

  AI · Sep 2026 · aina-tech.io