Dev tools · alternatives · 2026

24 alternatives to Offeryn – Build Tools for LLMs in Rust, with Model Context Protocol

Build tools for LLMs in Rust using Model Context Protocol - avahowell/offeryn

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
2
Building an LLM from Scratch in TypeScript▲33
The from-scratch guide to building large language models in NodeJS.
9d ago · book.humanexperiences.io · its alternatives →
3
LLM-Term – Simple Rust-based CLI assist tool▲32
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com · its alternatives →
4
Coerce – Actor runtime for Rust using async/await and tokio channels▲155
Actor runtime and distributed systems framework for Rust - LeonHartley/Coerce-rs
2019 · github.com · its alternatives →
5
Local LLMs by Sttabot AI▲112
Build local LLMs using top data science libraries
2023 · its alternatives →
6
An educational Local Qwen3 LLM Inference project written in Rust▲15
An educational Rust project for exporting and running inference on Qwen3 LLM family - reinterpretcat/qwen3-rs
2025 · github.com · its alternatives →
7
SwiftAI – open-source library to easily build LLM features on iOS/macOS▲74
We built SwiftAI, an open-source Swift library that lets you use Apple’s on-device LLMs when available (Apple opened access in June), and fall back to a cloud model when they aren’t available — all without duplicating code. SwiftAI gives you: - A single, model-agnostic API - An agent/tool loop - Strongly-typed structured outputs - Optional chat state Backstory: We started experimenting with Apple’s local models because they’re free (no API calls), private, and work offline. The problem: not all devices support them (older iPhones, Apple Intelligence disabled, low battery, etc.). That…
2025 · github.com · its alternatives →
8
Build AI DAGs with Memory; Run and Validate LLM Tools in Containers▲48
I am working on a modular open source framework called Griptape that allows Python developers to create LLM pipelines and DAGs for complex workflows that use rules and memory. Griptape can be thought of as "Airflow for LLMs," providing an alternative to the agent-based LangChain approach. Developers can also build reusable LLM tools with explicit JSON schemas that can be executed in any environment (local, containerized, cloud, etc.) and integrated into Griptape workflows. They can also be easily converted into ChatGPT Plugin APIs and LangChain tools via adapters. Tools can be thought of as…
2023 · github.com · its alternatives →
9
Learning Rust by writing a Markdown to HTML compiler▲25
Hi HN, I'm new to Rust and, after half of `The Rust Programming Language` I decided to delve into a project I would actually use: a markdown to html compiler that actually serves my micro-blog (I started just right after publishing the project hehe). The project is intentionally small and one-file contained: I wanted to have the full picture in mind. I'm open to comments, suggestions and PRs. If you asked me more convincing arguments to "why did you do it?" rather than "to learn Rust" I would respond with: LLMs are effectively a new type of web users today and will be more and more present…
Jul 2026 · andreadimatteo.com · its alternatives →
10
Arch-Router – 1.5B model for LLM routing by preferences, not benchmarks▲66
Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
2025 · its alternatives →
11
Use LLMs to Interact with APIs▲35
Recently featured in a LangChain blog https://blog.langchain.dev/empowering-development-with-flowt... , use LLMs to construct an API first runnable workflow with an IDE experience.
2024 · github.com · its alternatives →
12
Automated red teaming for your LLM app▲23
Hi HN, I built this open-source LLM red teaming tool based on my experience scaling LLMs at a big co to millions of users... and seeing all the bad things people did. How it works: - Uses an unaligned model to create toxic inputs - Runs these inputs through your app using different techniques: raw, prompt injection, and a chain-of-thought jailbreak that tries to re-frame the request to trick the LLM. - Probes a bunch of other failure cases (e.g. will your customer support bot recommend a competitor? Does it think it can process a refund when it can't? Will it leak your user's address?) -…
2024 · promptfoo.dev · its alternatives →
13
Toolkit for LLM Fine-Tuning, Ablating and Testing▲14
Hello all! Very happy to share this toolkit that allows you to fine-tune your choice of open-source LLMs on your data! The toolkit also allows you to run ablation studies across LLMs, prompt designs, training configurations, and can ingest different data files -- all through just ONE YAML file! After fine-tuning, you can also run a bunch of tests to ensure that the fine-tuned LLM behaves as expected, enabling faster time-to-production! Why this toolkit? Why now? While closed-source LLMs have become popular for chat-based applications, enterprises are considering a shift to self-hosted SLMs…
2024 · github.com · its alternatives →
14
I built an open-source Rust/TS AI agent runtime with a Next.js-style DX▲5
Link: https://docs.trysoma.ai/ For the past ~9 months I’ve been building Soma, an open-source AI agent & workflow runtime written in Rust, with a TypeScript SDK (Python coming soon). It’s not a framework; it’s meant to sit underneath whatever agent/tooling code you already write (Vercel AI SDK, LangChain, custom code, etc.). It provides features around your framework + a better DX for building agents. I’ve tried to take a Next.JS model: open-source, good DX, self-deployable. I originally set out to build a vertical back-office/operations product for SMEs. I needed a…
Dec 2025 · docs.trysoma.ai · its alternatives →
15
Engine Core – open-source LLM chat management and tool call framework▲24
Chat strategies for LLMs. Contribute to Engine-Labs/engine-core development by creating an account on GitHub.
2024 · github.com · its alternatives →
16
LLGTRT: TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output▲6
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com · its alternatives →
17KA
Knit – A Better LLM Playground▲14
Knit was created to solve pains of other LLM playgrounds. Some of the highlights: - Smart prompt builder, create prompt with simple requirement and few shot learning, fast and effortlessly. - Function call simulation, visualize the function callings and you can also setup a mocked value to return. - Support OpenAI/Anthropic/Azure models. - Manage prompts with projects and members. - And so much more! I have been developing Knit by myself for over 4 months now, and am looking for ways to improve it. Any feedback is appreciated.
2023 · promptknit.com · its alternatives →
18
mcp-golang – Build custom tools for LLMs in go▲5
Anthropic released the Model Context Protocol a couple weeks ago - https://www.anthropic.com/news/model-context-protocol. They have official sdks for python and typescript but nothing for go, so I wrote one! The main goal is to provide a low boilerplate sdk to write MCP servers, you can spin up a server which can give an LLM access to weather forecasts like this: https://github.com/metoro-io/mcp-golang/blob/main/examples/g... . Using it here: https://youtu.be/xaMkMl_R-0A The library handles generating all the MCP…
2024 · github.com · its alternatives →
19
Xybrid – run LLM and speech locally in your app (no back end, Rust)▲6
Hi HN, We built Xybrid, a Rust library for running LLM + speech pipelines directly inside your app, no server, no daemon, just one binary. We started building it while working on a privacy-focused LLM app with Tauri and realized there wasn’t a straightforward way to embed models directly into shipped applications without relying on a separate server process. Xybrid links into your process like any other library. It supports GGUF / ONNX / CoreML and integrates with Flutter, Swift, Kotlin, Unity, and Tauri, letting you run pipelines like speech → LLM → speech in a single call. On…
Mar 2026 · github.com · its alternatives →
20
Zero-dependency, TypeScript SDK for building robust NLP applications▲16
Hey HN! We just released a new library for building LLM-powered applications: @axflow/models. It is part of a larger suite of libraries we're developing for TypeScript developers working with generative AI. This library provides the simplest APIs for 1) invoking the most popular LLM and embedding models (openai, anthropic, cohere, huggingface, etc.) 2) streaming LLM responses to clients, including augmenting the streams with additional arbitrary data and 3) building client-side applications with React hooks. @axflow/models has zero dependencies and is built using only the…
2023 · docs.axflow.dev · its alternatives →
21
React-like Declarative DSL for building synthetic LLM datasets▲10
We were frustrated with dataset generation DX: ad‑hoc scripts and JSON templates break as flows branch, tool‑calls drift, and reproducibility is hard. So we built Torque: a schema‑first, declarative, fully typesafe DSL to generate conversational datasets. What it is: Declarative DSL - Compose conversations like React components Fully Typesafe - Zod schemas with complete type inference Provider Agnostic - Generate with any AI SDK provider (OpenAI, Anthropic, DeepSeek, vLLM, LLaMA.cpp etc.) AI-Powered Content - Generate realistic varied datasets automatically without complicated scripts Faker…
Nov 2025 · github.com · its alternatives →
22
Tired of building agents? throw an LLM at this framework▲14
Enable LLMs to Program Themselves. Contribute to Osly-AI/Pocket-Flow-Framework development by creating an account on GitHub.
2025 · github.com · its alternatives →
23
ADK-Rust▲6
Hey everyone! I'm excited to share ADK-Rust - a production-ready implementation of Google's Agent Development Kit in Rust. Why Rust? After working extensively with adk-python in developing an ai agent factory at zavora.ai, I wanted to bring the same powerful agent development patterns to the Rust ecosystem, targeting use cases where: Performance is critical - Rust's zero-cost abstractions and memory safety Deployment size matters - Single binary with no runtime dependencies Systems-level integration - Embedded systems, edge computing, IoT Concurrency at scale - Rust's async/await with…
Dec 2025 · adk-rust.com · its alternatives →
24
Composable middleware for LLM inference Optimization Passes▲7
Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…
Mar 2026 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →