Alternatives
Products that do what Ferrum does
Run local LLMs with one Rust binary
- 1
- 2FJ
2019 · github.com
- 3
- 4
Local sandboxes for AI agents on your Mac, Linux, bare metal
Aug 2026 · github.com
- 5
An ultra-fast, single-binary MCP server written in Rust as a lightweight alternative to Node.js/Python. - StamManif/mcp-stama
23d ago · github.com
- 6
- 7
- 8
- 9
A fast, rootless sandbox and virtual resource runtime for any workload, including untrusted and AI-generated code. Daemonless: a real, kernel-enforced container in ~3.5 ms from an OCI image, out of one static binary - getkern/kern
13d ago · github.com
- 10FR
FerroTerm is a Terminal Emulator (xterm/vt100) / Library witten in Rust and compiled to WebAssembly. It is approx 4x faster than xterm.js which is used by VSCode etc. and prodvides wide spec coverage. https://datanoisetv.github.io/ferroterm/ Would love some feedback and hear what people want to build with it.
Jul 2026
- 11LT
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com
- 12NA
Hi, over the past few months I've been working on this project: Numax is a small Rust runtime that does three things: it runs WebAssembly modules in a sandbox, has a built-in local key-value store, and syncs everything across nodes with CRDTs and gossip. Basically, you write a wasm module, run it on two machines, and they converge (I hope !). It's a decentralized system... I hope someone finds it interesting! There's a whitepaper I've put a lot into, and I think the code isn't bad either! I believe there's still room in this world for software that's fun and well made, and while building…
Jun 2026 · github.com
- 13TA
Hi HN! This Christmas I made `tysm`, a rust crates which simplifies using ChatGPT with Structured Outputs. Essentially you just tell it the rust type you want the output to conform to, and it handles creating the JSON Schema, passing it to ChatGPT's chat-completions API, and deserializing the response. Structured Outputs are very useful because they guarantee that the response will conform to whatever type you expect. I'm already finding my crate very useful since I love using Structured Outputs but I hate writing JSON Schemas myself. Technical details for rust heads: To generate the JSON…
2024 · github.com
- 14
One binary. HTTP, dataframes, Excel, S3 and AI built in.
Jul 2026 · docs-orion.onrender.com
- 15XR
Hi HN, We built Xybrid, a Rust library for running LLM + speech pipelines directly inside your app, no server, no daemon, just one binary. We started building it while working on a privacy-focused LLM app with Tauri and realized there wasn’t a straightforward way to embed models directly into shipped applications without relying on a separate server process. Xybrid links into your process like any other library. It supports GGUF / ONNX / CoreML and integrates with Flutter, Swift, Kotlin, Unity, and Tauri, letting you run pipelines like speech → LLM → speech in a single call. On…
Mar 2026 · github.com
- 16HA
Demo starts at 50m into the video. This was a bit terrifying to record because 2am the previous night everything was totally broken after a major refactor (so that we could add external LLM support as well as local GPUs). But pressure can be a useful force :-D We start with a stack deployed on my laptop without a GPU, pointing to together.ai so we can run open source LLMs easily without having to have access to a GPU. We show simple inference through the ChatGPT-like web interface (with users, sessions etc) and then simple drag'n'drop RAG. Then we show some helix apps defined as yaml: Marvin…
2024 · youtube.com
- 17IB
Link: https://docs.trysoma.ai/ For the past ~9 months I’ve been building Soma, an open-source AI agent & workflow runtime written in Rust, with a TypeScript SDK (Python coming soon). It’s not a framework; it’s meant to sit underneath whatever agent/tooling code you already write (Vercel AI SDK, LangChain, custom code, etc.). It provides features around your framework + a better DX for building agents. I’ve tried to take a Next.JS model: open-source, good DX, self-deployable. I originally set out to build a vertical back-office/operations product for SMEs. I needed a…
Dec 2025 · docs.trysoma.ai
- 18RC
I've been working on Red Candle, a Ruby gem that runs LLMs (Llama, Mistral, Gemma, Phi) directly in your Ruby process through Rust bindings (based on the candle crate from Hugging Face). No Python, no servers - just FFI with Metal/CUDA acceleration. It's been useful for adding AI features to Rails apps without the complexity of managing separate services. Would love feedback from anyone working with LLMs in Ruby.
2025 · github.com
- 19FR
2019 · github.com
- 20
- 21
Open-source VMs-as-a-service. Contribute to stanford-mast/blast development by creating an account on GitHub.
9d ago · github.com
- 22OS
I built a network discovery tool in Rust that helps identify public Ollama LLM servers. It scans IP ranges to find Ollama instances and catalogs their available models. Important note: This is intended for educational purposes and authorized security testing only.
2025
- 23MR
Hey HN, With the news of OpenRouter possibly being acquired and proliferation of hosted LLM routers (i.e. Ramp Router, Vercel’s AI Gateway), I saw the need for a self hosted solution focused on cost savings, transparency, and performance. So, I built an open sourced router with a simple CLI interface that can easily sit between coding agents and GenAI workloads. For the curious and lazy, at the moment, Millwright has the tools for, - Providers: OpenAI-compatible APIs, Anthropic, Amazon Bedrock - Routing: policy-controlled model roles (cheap, mid, frontier), cheapest healthy route selection -…
Jul 2026 · github.com
- 24CR
Clawbernetes turns OpenClaw into an AI-native infrastructure manager. Instead of YAML, Helm charts, and kubectl — you have a conversation. "Deploy Llama 70B on the node with the most VRAM" → agent selects the best node, pulls the image, starts the container with GPU passthrough, sets up health monitoring. "Why is inference slow?" → checks GPU temps, VRAM, CPU load. "GPU 0 at 89°C — thermal throttling. Want me to reduce batch size?" 23 crates, 74K lines of Rust, 1,866 tests, zero unsafe in core. Supports CUDA, Metal, ROCm, Vulkan, and CPU SIMD. Components: - clawnode: node agent with 80+…
Feb 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →