Alternatives

Products that do what Run Llama.cpp In-Process from Java with Project Panama FFM does

1
Ollama▲235
The easiest way to run large language models locally
2023
2
Llama▲91
A terminal file manager
2021
3
Llama MacOS Desktop Controller▲102
Let Llama take over your desktop
2025
4RP
Run Python, Ruby, Node.js, C++, Lua in the Browser via x86 to WASM JIT▲171
2021 · repl.leaningtech.com
5RL
Run LLM generated scripts in CLI.▲5
2023 · verbomate.com
6
Kuzco▲216
Open-source Swift package to run LLMs locally on iOS & macOS
2025
7
Code Llama 70B▲144
New, performant version of Meta's LLM for code generation
2024
8DM
Docker Model Runner Integrates vLLM for High-Throughput Inference▲7
Hi HN, I’m one of the authors of this post. We’ve updated Docker Model Runner to support vLLM alongside the existing llama.cpp backend. The goal is to bridge the gap between local prototyping (often done with GGUF/llama.cpp) and high-throughput production (often done with Safetensors/vLLM) using a consistent Docker workflow. Key technical details: Auto-routing: The tool detects the model format. If you pull a GGUF model, it routes to llama.cpp. If you pull a Safetensors model, it routes to vLLM. API: It exposes an OpenAI-compatible API (/v1/chat/completions), so the…
Nov 2025 · github.com
9RP
Run Puppeteer on AWS Lambda▲75
2018 · github.com
10LA
Llamero – A GUI app to easily download, install and infer LLaMA models▲7
2023 · github.com
11LA
Llama – a fun, timed, task manager▲6
2020 · llamalife.co
12CF
CLI for executing code in many different languages with Docker▲64
2015 · docker-exec.github.io
13PT
Pascal to LLVM Compiler in Haskell▲61
2020 · github.com
14GE
Game Engine Using LLVM and Ruby (XPlat Including WASM)▲11
2019 · fiddle.dragonruby.org
15AL
An LLM Running on a PS Vita▲5
Hello all, a couple of moons ago I ported karpathy's llama2.c code to run inference on the TinyStories 260K & 15M checkpoints on the on the PS Vita with the ability to download/delete the models on device. Runs showed that the 260K model ran at ~120 tok/s and at 15M ran at 1.8 tok/s, which probably could be a bit higher if it weren't a single threaded application. Had fun working on it as a weekend project, check it out for yourselves: https://github.com/callbacked/psvita-llm
2025 · github.com
16CL
Chaos Llama – Chaos Monkey Build on AWS Lambda▲12
2016 · github.com
17LS
LambdaPi – Serverless Runtime Environment for LLM-Generated Code▲6
2023 · github.com
18LR
Lang: run .c .cpp .py .cljs .ts and more directly in Chrome offline▲5
2022 · github.com
19LP
LLM prompt template to build backend APIs faster▲7
2025 · codehooks.io
20LN
Llmpm – NPM for LLMs▲6
npm for LLMs — install, run, and share AI models. We’ve built llmpm, a CLI tool that makes open-source LLMs installable like packages. llmpm install llama3 llmpm run llama3 You can also package models with your projects so others can reproduce the same setup easily. Website: https://llmpm.co GitHub:https://github.com/llmpm/llmpm-dev
Mar 2026 · llmpm.co
21IB
I built an open-source answer engine that runs local LLMs▲5
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
22AS
A static scanner for LLM app code▲6
2025 · github.com
23AO
Airgapped Offline RAG – Run LLMs Locally with Llama, Mistral, & Gemini▲9
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com
24IL
Integrate LLM in Your Shell▲7
2025 · github.com

Ranked by how close each launch is in meaning, then by votes. Refine with a description →