# TrnRT alternatives

TrnRT: The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.

24 products that do a similar job, closest first. 20 have pricing checked on their own site; 19 of those are free or have a free tier.

Page: https://nowfound.app/alternatives/trnrt-an-open-source-llm-inference-engine-for-aws-trainium
About TrnRT: https://nowfound.app/launch/trnrt-an-open-source-llm-inference-engine-for-aws-trainium.md

- [Llama 3.2 Interpretability with Sparse](https://nowfound.app/launch/llama-3-2-interpretability-with-sparse-autoencoders): This project replicates mechanistic interpretability research on large language models using sparse autoencoders, building on published work from Anthropic, OpenAI, and DeepMind. (2024-11-21, 579 votes, free, github.com)
- [LLM Gateway](https://nowfound.app/launch/llm-gateway): LLM Gateway is an open-source API platform that lets users access multiple AI models through a single unified interface. (2025-07-03, 286 votes, free tier, llmgateway.io)
- [LLM](https://nowfound.app/launch/llm-a-rust-crate-cli-for-cpu-inference-of-llms-llama-gpt-neo): LLM is an open-source Rust crate and command-line tool for running large language models like LLaMA and GPT-NeoX on CPUs. (2023-05-09, 45 votes, free, github.com)
- [DB-GPT](https://nowfound.app/launch/db-gpt-an-llm-tool-for-database): open-source agentic AI data assistant for the next generation of AI + Data products. (2023-05-23, 156 votes, free, github.com)
- [LocalGPT](https://nowfound.app/launch/localgpt-a-local-first-ai-assistant-in-rust-with-persistent): LocalGPT is a Rust-based AI assistant that runs locally as a single 27MB binary without requiring Node.js, Docker, or Python. (2026-02-08, 331 votes, free, github.com)
- [DeepSeek-R1-0528](https://nowfound.app/launch/deepseek-r1-0528): New open-source LLM that rivals o3 in coding & reasoning (2025-05-29, 218 votes, from $9/mo, huggingface.co)
- [uzu](https://nowfound.app/launch/we-made-our-own-inference-engine-for-apple-silicon): This is a Rust-based inference engine designed for Apple Silicon, built from scratch to run large language models. (2025-07-15, 186 votes, free, github.com)
- [traceAI](https://nowfound.app/launch/traceai): Open-source LLM tracing that speaks GenAI, not HTTP. (2026-04-01, 263 votes, free, github.com)
- [Inference Engine by GMI Cloud](https://nowfound.app/launch/inference-engine-by-gmi-cloud): Fast multimodal-native inference at scale (2025-12-07, 180 votes, gmicloud.ai)
- [Klarity](https://nowfound.app/launch/klarity-open-source-tool-to-analyze-uncertainty-entropy-in-l): We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertaint… (2025-02-03, 132 votes, free, github.com)
- [Unify](https://nowfound.app/launch/unify-dynamic-llm-benchmarks-and-sso-for-multi-vendor-deploy): Unify provides a Model Hub that tracks live performance metrics for 21 large language models from providers including OpenAI, Mistral AI, and Together AI. (2024-02-06, 91 votes)
- [Tiny-vLLM](https://nowfound.app/launch/tiny-vllm-high-performance-llm-inference-engine-in-c-and-cud): Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM (2026-05-29, 205 votes, free, github.com)
- [Dolly](https://nowfound.app/launch/dolly): Democratizing the magic of ChatGPT with open models (2023-03-27, 113 votes, free, github.com)
- [An educational Local Qwen3 LLM Inference](https://nowfound.app/launch/an-educational-local-qwen3-llm-inference-project-written-in): An educational Rust project for exporting and running inference on Qwen3 LLM family (2025-07-12, 15 votes, free, github.com)
- [A tool to benchmark LLM APIs (OpenAI, Claude](https://nowfound.app/launch/a-tool-to-benchmark-llm-apis-openai-claude-local-self-hosted): This open-source tool benchmarks different LLM APIs including OpenAI, Claude, and self-hosted models like llama.cpp. (2025-06-29, 55 votes, llmapitest.com)
- [LLGTRT](https://nowfound.app/launch/llgtrt-tensorrt-llm-rust-server-w-openai-compat-and-structur): TensorRT-LLM server with Structured Outputs (JSON) built with Rust (2024-10-30, 6 votes, free, github.com)
- [Taylor AI](https://nowfound.app/launch/taylor-ai): Fine-tune open source LLMs in minutes (2023-08-09, 118 votes, trytaylor.ai)
- [LLM-Term](https://nowfound.app/launch/llm-term-simple-rust-based-cli-assist-tool): LLM-Term is a Rust-based CLI tool that provides AI assistance directly from the command line. (2024-08-29, 32 votes, free, github.com)
- [Improve Text-to-SQL Accuracy with LLM](https://nowfound.app/launch/improve-text-to-sql-accuracy-with-llm): A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL (2023-07-11, 17 votes, free, github.com)
- [Pip install inference, open source computer](https://nowfound.app/launch/pip-install-inference-open-source-computer-vision-deployment): Inference is an open source computer vision deployment tool that simplifies running vision models on various hardware through a standardized HTTP-based interface. (2023-08-23, 71 votes, free, github.com)
- [Superflows](https://nowfound.app/launch/superflows-open-source-ai-copilot-for-saas-products): Henry, Matt and James here – we’re building an open source toolkit that makes it easy to integrate an LLM-powered copilot that talks to your API into software products. It works by calling API endpoi… (2023-09-16, 37 votes, free, github.com)
- [Entropy-Guided Loop](https://nowfound.app/launch/entropy-guided-loop-how-to-make-small-models-reason): TLDR: A small, vendor-agnostic inference loop that turns token logprobs/perplexity/entropy into an extra pass and reasoning for LLMs. - Captures logprobs/top-k during generation, computes perplexity… (2025-09-03, 33 votes, free, github.com)
- [Engine](https://nowfound.app/launch/engine-a-multi-llm-alternative-to-codex): Please forgive the sign up. It means we can give away and keep track of free inference usage/no API keys needed. Happy to send sign in credentials on request if you prefer. (2025-05-20, 16 votes, free tier, enginelabs.ai)
- [Free Inference Engineer and Model Training](https://nowfound.app/launch/free-inference-engineer-and-model-training-roadmap): Free, gamified roadmaps for LLM engineering: an Inference Engineering path (KV caches, CUDA kernels, production vLLM serving) and a Model Training path (pretraining on a budget, scaling laws, SFT/DPO… (2026-08-24, 16 votes, free, inferquest.org)
