Alternatives
Products that do what Pantry – Brew/Docker for LLMs does
Started this a few months ago because I wanted a better test runner for an entirely different LLM-based project, then got completely nerdsniped by making LLMs as easy as possible to hotswap into other projects. It's still on the early side but it's finally at the point where I would happily use it for my original project, so I figured I'd post it. It's also—almost by accident—fully remote compatible (permissioning system included!), so you can host it on a box and then connect a program to it remotely.
- 1OR
Hi HN A few folks and I have been working on this project for a couple weeks now. After previously working on the Docker project for a number of years (both on the container runtime and image registry side), the recent rise in open source language models made us think something similar needed to exist for large language models too. While not exactly the same as running linux containers, running LLMs shares quite a few of the same challenges. There are "base layers" (e.g. models like Llama 2), specific configuration to run correctly (parameters, temperature, context window sizes etc). There's…
2023 · github.com
- 2MG
Hello HN, I've been working on this project for a while, and it has been in an "open" beta for some time. I finally believe it's ready for its first release. I hope you like it. Here are some potential questions that may arise: 1. How does it compare to LM Studio? It's likely that if you're already using LM Studio, you'll continue to do so. This project is designed to be more user-friendly. 2. Is it open-source? No, it is not. 3. Does it use any open-source libraries? Yes, it uses llama.cpp and a few others, as indicated in the license information included with the application. 4. Why is not…
2023 · avapls.com
- 3YA
Built this for my LLM workflows - needed searchable, persistent memory that wouldn't blow up storage costs. I also wanted to use it locally for my research. It's a content-addressed storage system with block-level deduplication (saves 30-40% on typical codebases). I have integrated the CLI tool into most of my workflows in Zed, Claude Code, and Cursor, and I provide the prompt I'm currently using in the repo. The project is in C++ and the build system is rough around the edges but is tested on macOS and Ubuntu 24.04.
2025 · github.com
- 4BA
Bash4LLM is a single-file Bash wrapper for interacting with LLMs from the terminal. I created it because I wanted something simple that worked without installing Python, Node, or any other runtime. It uses only Bash, curl, and jq. You can send prompts, start a small chat, process files line by line, stream output, and save session metadata in JSON format. I tried to make it safe and predictable: no use of the system /tmp, no use of eval. Groq is supported by default, and other providers can be added with dedicated Bash scripts in the extras/providers/ folder. Example: echo…
Jun 2026 · github.com
- 5RM
Dec 2025 · github.com
- 6RL
I've been looking for a way to run LLMs safely without needing to approve every command. There are plenty of projects out there that run the agent in docker, but they don't always contain the dependencies that I need. Then it struck me. I already define project dependencies with mise. What if we could build a container on the fly for any project by reading the mise config? I've been using agent-en-place for a couple of weeks now, and it's working great! I'd love to hear what y'all think
Jan 2026 · github.com
- 7LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com
- 8KA
Knit was created to solve pains of other LLM playgrounds. Some of the highlights: - Smart prompt builder, create prompt with simple requirement and few shot learning, fast and effortlessly. - Function call simulation, visualize the function callings and you can also setup a mocked value to return. - Support OpenAI/Anthropic/Azure models. - Manage prompts with projects and members. - And so much more! I have been developing Knit by myself for over 4 months now, and am looking for ways to improve it. Any feedback is appreciated.
2023 · promptknit.com
- 9DM
Hey Hacker News, We're the maintainers of docker/model-runner and wanted to share some major updates we're excited about. Link: https://github.com/docker/model-runner We are rebooting the community: https://www.docker.com/blog/rebooting-model-runner-community... At its core, model-runner is a simple, backend-agnostic tool for downloading and running local large language models. Think of it as a consistent interface to interact with different model backends. One of our main backends is llama.cpp, and we make it a point to contribute any…
Oct 2025 · github.com
- 10LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com
- 11IB
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
- 12PU
After seeing a cool demo of a hack on Twitter, I built a cross platform version of it that works well and uses streaming. From anywhere on Mac and Linux, trigger Ollama and optionally feed it your clipboard. I built it yesterday and it's already very useful to me. I'm pretty excited about it and wanted to share!
2024 · github.com
- 13UL
Hi everyone, Just wanted to share a use case where local LLMs are genuinely helpful for daily workflows: file organization. I've been working on a C++ desktop app called AI File Sorter – it uses local LLMs via `llama.cpp` to help organize messy folders like `Downloads` or `Desktop`. Not sort files into folders solely based on extension or filename patterns, but based on what each file actually is supposed to do or does. Basically: what would normally take me a great deal of time for dragging and sorting can now be done in a few. It's cross-platform (Windows/macOS/Linux), and fully…
2025 · github.com
- 14CK
Hi HN, for quite some time I've been thinking how LLMs are missing the knowledge base, where I can dump CSVs, PDFs, and most important, inline web app. running on Claude Code (bring your own agent) with agents with heartbeats and jobs https://runcabinet.com It runs locally and is installable via npm. GitHub (open source): https://github.com/hilash/cabinet This is still very early. I put the first version together quickly after seeing a post by Andrej Karpathy about LLM knowledge bases, which matched closely with what I’d been building. Some people have already…
Apr 2026 · runcabinet.com
- 15LV
Hello HN! We've been working on a new hypervisor https://kwarantine.xyz that can run strongly isolated containers. This is still a WIP, but we wanted to give the community an idea about our approach, its benefits, and various use cases it unlocks. Today, VMs are used to host containers, and make up for the lack of strong security as well as kernel isolation in containers. This work adds this missing security piece in containers. We plan on launching a free private beta soon. Meanwhile, we'd deeply appreciate any feedback, and happy to answer any questions here or on our slack…
2021
- 16KP
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com
- 17OS
And you can try out the models live here: https://labs.refuel.ai/playground
2024 · huggingface.co
- 18IB
After fine-tuning GPT for a personal project, I realized how tedious it is to write plain text in a massive JSON file. That's why I built this app for my own use, and I want to see if others could benefit from a tool like this as well ;)
2024 · finetuna-ui.com
- 19CR
hi everyone. how does moving llm call prompts and output structure definitions away from code into configuration land sound? would you use something like this if it was stable and well documented enough? please don't hold back the criticism. i appreciate all feedback (constructive & otherwise).
2024 · github.com
- 20IL
2025 · github.com
- 21EL
Hey HN! I built Experiment to solve a common frustration in LLM development: the lack of proper tools for prompt engineering experimentation. Here's what makes it different: Key Features: - Load and edit chat completion logs from CSV files - Fork and modify specific conversation entries - Run inference via Anthropic, Mistral, and OpenAI - Define custom tools using JSONSchema format - Visual tool usage analysis with collapsible, sorted key-value pairs - Full mobile support and available as installable PWA Technical Highlights: - Built with React using custom isomorphic architecture -…
2025 · github.com
- 22IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 23CF
Over this past month I had the idea to build a 100% open-source MIT-licensed tool to simplify sharing code with LLMs, without the vendor lock-in you get from most SDKs. Right now, it’s way too hard to export your data or work freely with models like o1 PRO or Grok 3, especially since they don’t even have API access. So I built OpenRepoPrompt, an open-source tool from wildberry-source that serializes files and folders into XML for LLMs. I coded/designed from 12PM -> 11PM on Saturday and 8AM -> 11PM on Sunday. There are still tons of features missing (I'm working on better file filtering…
2025 · github.com
- 24LC
Hey, folks here is a peek into Jujutsu. We at Poozle are working with hundreds of APIs and it has been always frustrating to 1. Search the API in the documentation or ask ChatGPT 2. Then copy it to the postman and understand/test the API 3. Generate code to integrate into the codebase We thought how about having all of this at one place. We currently fine-tuned LLM on public REST APIs to reduce hallucination and then combined it with ChatGPT and Postman. I look forward to feedback, feature requests and discussions!
2023 · loom.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →