AI · alternatives · 2026

24 alternatives to Strot – Compile an LLM's intent once, run decisions locally forever

STROT (arXiv:2505.01636): compile-time LLM code synthesis with deterministic, self-routing NLU runtimes. PyPI: strot-ai - strot-ai/Project-STROT

Strot is an AI tool that uses large language models to generate code once during a compile phase, then runs natural language understanding decisions locally without requiring further LLM calls. It features… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 2 we have checked on their own sites, 2 of them free or with a free tier.

1
FastRouter.aifree tier, from $199/mo▲364
Route requests to the right LLM for cost, latency & quality
23d ago · fastrouter.ai · its alternatives →
2
LLM Gatewayfree tier▲286
I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
3
Arch▲314
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
2024 · github.com · its alternatives →
4
Continue – Open-source coding autopilot▲298
Hi HN, we’re Nate and Ty, co-founders of Continue, an open-source autopilot for software development built to be deeply customizable and continuously learn from development data. It consists of an extended language server and (to start) a VS Code extension. Our GitHub is https://github.com/continuedev/continue. You can watch a demo of Continue and download the extension at https://continue.dev — — — A growing number of developers are replacing Google + Stack Overflow with Large Language Models (LLMs) as their primary approach to get help, similar to how…
2023 · github.com · its alternatives →
5
AI Playground by Vercel Labs▲249
Hey, Jared Palmer (creator of this playground) here. Really excited to ship this. I’ve been building this over the past few weeks to compare LLMs from different providers like OpenAI, Anthropic, Cohere, etc. At Vercel, I manage our Frameworks division (including Next.js, Svelte, and Turbo) and wanted to also dogfood some of the latest features in a slightly larger application. This playground takes a lot of inspiration from https://nat.dev and is built on Tailwind, ui.shadcn.com, and some upcoming Vercel products we’re announcing soon. We’re going to continue adding models to…
2023 · play.vercel.ai · its alternatives →
6
A weekend replication of STOKE, a stochastic superoptimiser▲58
Haskell re-implementation of STOKE, the stochastic superoptimizer - blaze/notebooks/tutorial.ipynb at master · bollu/blaze
2019 · github.com · its alternatives →
7
Local LLMs by Sttabot AI▲112
Build local LLMs using top data science libraries
2023 · its alternatives →
8
Tiny-vLLM – high performance LLM inference engine in C++ and CUDA▲205
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
9
Only 1 LLM can fly a drone▲180
📸 gotta find 'em all; spatial reasoning benchmark for LLMs - kxzk/snapbench
Jan 2026 · github.com · its alternatives →
10
SelfHostLLM▲134
A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
11
Pipelex – Declarative language for repeatable AI workflows▲122
We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
Oct 2025 · github.com · its alternatives →
12
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
13
Aster by AsterWise▲65
Intelligent model routing for code, agents and workflows
22d ago · asterwise.dev · its alternatives →
14
Arch-Router – 1.5B model for LLM routing by preferences, not benchmarks▲66
Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
2025 · its alternatives →
15
20+ Claude Code agents coordinating on real work (open source)▲53
Single-agent LLMs suck at long-running complex tasks. We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress. How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents We tested this on a Putnam-level math problem, but the…
Feb 2026 · github.com · its alternatives →
16
LLM-Term – Simple Rust-based CLI assist tool▲32
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com · its alternatives →
17
Open-source AI studio – Core infrastructure stack for LLM Apps▲29
[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications - pyadav/gateway
2024 · github.com · its alternatives →
18
An educational Local Qwen3 LLM Inference project written in Rust▲15
An educational Rust project for exporting and running inference on Qwen3 LLM family - reinterpretcat/qwen3-rs
2025 · github.com · its alternatives →
19
Local LLM code-generation with Gemma 4 e2B via JSON AST to Clojure▲22
JSON AST > Clojure. Contribute to quadracollision/llmisp development by creating an account on GitHub.
May 2026 · github.com · its alternatives →
20
Engine – A multi-LLM alternative to Codex▲16
Please forgive the sign up. It means we can give away and keep track of free inference usage/no API keys needed. Happy to send sign in credentials on request if you prefer.
2025 · enginelabs.ai · its alternatives →
21
Use LLMs to Interact with APIs▲35
Recently featured in a LangChain blog https://blog.langchain.dev/empowering-development-with-flowt... , use LLMs to construct an API first runnable workflow with an IDE experience.
2024 · github.com · its alternatives →
22
Composable middleware for LLM inference Optimization Passes▲7
Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…
Mar 2026 · github.com · its alternatives →
23
Outline Driven Development – New AI-Assisted Coding Paradigm; BN▲11
The Problem "Vibing" with LLMs is often too shallow for complex logic, while writing full specifications is cognitively expensive and slow. We need a middle ground that mimics how human programmers gather context—scanning structure before diving into details. The Solution: Outline Driven Development (ODD) I've built a "batteries-included" kit for Gemini/Claude/Codex that uses AST analysis to understand code structure rather than just raw text. This relies on a hyper-optimized Rust toolchain (`ast-grep`, `ripgrep`, `jj`, etc.) to feed precise, structural context to the agent. 1. The…
Nov 2025 · github.com · its alternatives →
24
An application stack Claude coded directly in LLVM IR▲10
This repo is the result of a debate about what kind of programming language might be appropriate if humans are no longer the primary authors. Initially the thought was "LLMs can just generate binaries directly" (this was before a more famous person had the same idea). But that on reflection seems like a bad approach because languages exist to capture program semantics that are elided by translation to machine code. The next step was to wonder if an existing "machine readable" program representation can be the target for LLM code generation. It turns out yes. This project is the result of…
Mar 2026 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →