AI · alternatives · 2026
24 alternatives to TrnRT
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
TrnRT is an open-source inference engine designed for running large language models on AWS Trainium hardware. It features NKI megakernels for optimized performance, supports context windows up to 1 million tokens, and… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 20 we have checked on their own sites, 19 of them free or with a free tier.
Only the free ones: free alternatives to TrnRT →
- 1

This project replicates mechanistic interpretability research on large language models using sparse autoencoders, building on published work from Anthropic, OpenAI, and DeepMind.
2024 · github.com · its alternatives →
- 2
LLM Gatewayfree tier▲286LLM Gateway is an open-source API platform that lets users access multiple AI models through a single unified interface.
2025 · llmgateway.io · its alternatives →
- 3
LLMfree▲45LLM is an open-source Rust crate and command-line tool for running large language models like LLaMA and GPT-NeoX on CPUs.
2023 · github.com · its alternatives →
- 4
DB-GPTfree▲156open-source agentic AI data assistant for the next generation of AI + Data products.
2023 · github.com · its alternatives →
- 5
LocalGPTfree▲331LocalGPT is a Rust-based AI assistant that runs locally as a single 27MB binary without requiring Node.js, Docker, or Python.
Feb 2026 · github.com · its alternatives →
- 6
DeepSeek-R1-0528from $9/mo▲218New open-source LLM that rivals o3 in coding & reasoning
2025 · huggingface.co · its alternatives →
- 7
uzufree▲186This is a Rust-based inference engine designed for Apple Silicon, built from scratch to run large language models.
2025 · github.com · its alternatives →
- 8
traceAIfree▲263Open-source LLM tracing that speaks GenAI, not HTTP.
Apr 2026 · github.com · its alternatives →
- 9

Fast multimodal-native inference at scale
Dec 2025 · gmicloud.ai · its alternatives →
- 10
Klarityfree▲132We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty. What Klarity does: - Real-time analysis of model uncertainty during generation - Dual analysis combining log probabilities and semantic understanding - Structured JSON output with actionable insights - Fully self-hostable with customizable analysis models The tool works by analyzing each step of text generation and returns a structured JSON: - uncertainty_points: array of {step, entropy,…
2025 · github.com · its alternatives →
- 11UUnify▲91
Unify provides a Model Hub that tracks live performance metrics for 21 large language models from providers including OpenAI, Mistral AI, and Together AI.
2024 · its alternatives →
- 12
Tiny-vLLMfree▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
May 2026 · github.com · its alternatives →
- 13
Dollyfree▲113Democratizing the magic of ChatGPT with open models
2023 · github.com · its alternatives →
- 14

An educational Rust project for exporting and running inference on Qwen3 LLM family
2025 · github.com · its alternatives →
- 15AT
This open-source tool benchmarks different LLM APIs including OpenAI, Claude, and self-hosted models like llama.cpp.
2025 · llmapitest.com · its alternatives →
- 16
LLGTRTfree▲6TensorRT-LLM server with Structured Outputs (JSON) built with Rust
2024 · github.com · its alternatives →
- 17

- 18
LLM-Termfree▲32LLM-Term is a Rust-based CLI tool that provides AI assistance directly from the command line.
2024 · github.com · its alternatives →
- 19

A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL
2023 · github.com · its alternatives →
- 20

Inference is an open source computer vision deployment tool that simplifies running vision models on various hardware through a standardized HTTP-based interface.
2023 · github.com · its alternatives →
- 21
Superflowsfree▲37Henry, Matt and James here – we’re building an open source toolkit that makes it easy to integrate an LLM-powered copilot that talks to your API into software products. It works by calling API endpoints which you choose to expose to it. This lets the chatbot complete tasks within your software in response to natural language queries. It’s also open source, so you don’t have to send user data to another 3rd party. We support Llama 2, but we haven’t fine-tuned Llama 2 yet (coming soon) so highest accuracy is seen with GPT-4 or fine-tuned GPT-3.5 (much faster). We started working together 2…
2023 · github.com · its alternatives →
- 22
Entropy-Guided Loopfree▲33TLDR: A small, vendor-agnostic inference loop that turns token logprobs/perplexity/entropy into an extra pass and reasoning for LLMs. - Captures logprobs/top-k during generation, computes perplexity and token-level entropy. - Triggers at most one refine when simple thresholds fire; passes a compact “uncertainty report” (uncertain tokens + top-k alts + local context) back to the model. - In our tests on technical Q&A / math / code, a small model recovered much of “reasoning” quality at ~⅓ the cost while refining ~⅓ of outputs. I kept seeing “reasoning” models behave…
2025 · github.com · its alternatives →
- 23
Enginefree tier▲16Please forgive the sign up. It means we can give away and keep track of free inference usage/no API keys needed. Happy to send sign in credentials on request if you prefer.
2025 · enginelabs.ai · its alternatives →
- 24

Free, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO/GRPO) — 185 tasks with auto-verified milestones instead of a paper certificate.
Aug 2026 · inferquest.org · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →