# TrnRT

> The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.

Page: https://nowfound.app/launch/trnrt-an-open-source-llm-inference-engine-for-aws-trainium

- Launched: 2026-11-17
- Launch-day votes: 54, 8 comments; #503 of 3,759 launches that month (top 13%)
- Website: [github.com](https://github.com/RunanywhereAI/trnrt), online when checked 2026-11-17
- Category: [AI](https://nowfound.app/c/ai.md)
- Pricing: Free. Free and open source (checked 2026-11-20 on its site)

## In plain words

TrnRT is an open-source inference engine designed for running large language models on AWS Trainium hardware. It features NKI megakernels for optimized performance, supports context windows up to 1 million tokens, and provides an OpenAI-compatible API for easy integration. The tool is intended for developers and organizations looking to deploy LLMs efficiently on AWS infrastructure.

## What it does, from its own words

The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere. - RunanywhereAI/trnrt

## Does a similar job

- [LLM Gateway](https://nowfound.app/launch/llm-gateway): LLM Gateway is an open-source API platform that lets users access multiple AI models through a single unified interface. (2025-07-03, 286 votes, free tier, llmgateway.io)
- [LLM](https://nowfound.app/launch/llm-a-rust-crate-cli-for-cpu-inference-of-llms-llama-gpt-neo): LLM is an open-source Rust crate and command-line tool for running large language models like LLaMA and GPT-NeoX on CPUs. (2023-05-09, 45 votes, free, github.com)
- [DB-GPT](https://nowfound.app/launch/db-gpt-an-llm-tool-for-database): open-source agentic AI data assistant for the next generation of AI + Data products. (2023-05-23, 156 votes, free, github.com)
- [DeepSeek-R1-0528](https://nowfound.app/launch/deepseek-r1-0528): New open-source LLM that rivals o3 in coding & reasoning (2025-05-29, 218 votes, from $9/mo, huggingface.co)
- [uzu](https://nowfound.app/launch/we-made-our-own-inference-engine-for-apple-silicon): This is a Rust-based inference engine designed for Apple Silicon, built from scratch to run large language models. (2025-07-15, 186 votes, free, github.com)
- [Inference Engine by GMI Cloud](https://nowfound.app/launch/inference-engine-by-gmi-cloud): Fast multimodal-native inference at scale (2025-12-07, 180 votes, gmicloud.ai)

All alternatives: https://nowfound.app/alternatives/trnrt-an-open-source-llm-inference-engine-for-aws-trainium.md

## For the maker of TrnRT

- Feature it at the top of the NowFound home page for $19 for 30 days, marked sponsored: https://nowfound.app/feature/trnrt-an-open-source-llm-inference-engine-for-aws-trainium
- Free: put the badge on the product's own site and it joins Featured products after the paid spots. HTML for the footer: `<a href="https://nowfound.app/launch/trnrt-an-open-source-llm-inference-engine-for-aws-trainium"><img src="https://nowfound.app/badge/trnrt-an-open-source-llm-inference-engine-for-aws-trainium.svg" alt="TrnRT on NowFound" height="32"></a>`
