Alternatives
Products that do what Ergonomically call LLM in bulk from CLI does
Hi! I've found myself repeatedly writing little scripts to do bulk calls to LLMs for various tasks. For example, run some analysis on a large list of records. There are a few "gotchas" to doing this. For example, some service providers have rate limits, and some models will not reliably return JSON (if you're asking for it). So, I've written a command for this. What I've tried to do here is let the user break up prompts and configuration as they see fit. For example, you can have a prompt file which includes the API key, rate limit, settings, etc. all together, or break these up into…
- 1
- 2
- 3LB
For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…
2022 · github.com
- 4KC
I think in-process key management is the right abstraction for multi-key LLM setups. Not LiteLLM, not a Redis queue, not a custom load balancer. The failure modes are well-understood: a key gets rate-limited, you wait, you try the next one. Billing errors need a longer cooldown than rate limits. This is not a distributed systems problem — it's a state machine that fits in a library. The problem is everyone keeps solving it with infrastructure instead. Spin up LiteLLM, now you have a Python service to maintain. Reach for Redis, now you have a database for a problem that doesn't need one.…
Mar 2026 · github.com
- 5CL
I tried to port LLMLingua-2's official Python implementation into TypeScript. For best performance, open the URL with a WebGPU enabled web browser. Learn More: https://github.com/atjsh/llmlingua-2-js
2025 · atjsh.github.io
- 6EA
A few months ago I was working on a flight search engine that would include pet transport costs (I know a few by hearth but storing them and make the calculations in the UI would be nice) While I was collecting pet pricing from several airlines I strugled to extract data in a common format without hallucinated values. That's when I thought: What if I use multiple LLMs and take the most common response to improve accuracy? This idea became this new project. You provide your documents, an SQLModel schema, an LLM provider, plus what you'd like to extract and Extrai does the rest. Including…
Nov 2025 · github.com
- 7CR
hi everyone. how does moving llm call prompts and output structure definitions away from code into configuration land sound? would you use something like this if it was stable and well documented enough? please don't hold back the criticism. i appreciate all feedback (constructive & otherwise).
2024 · github.com
- 8TL
Little tool that I made to understand how (un)reasonable my prompts are.
Jan 2026 · github.com
- 9RL
2023 · verbomate.com
- 10A1
I've seen a lot of comments about how complex frameworks like LangChain can be. Over the holidays, I wanted to see how minimal an LLM framework could get if we stripped away everything non-essential. The result is an LLM framework in just 100 lines of code. These 100 lines capture what I see as the core abstraction of most LLM frameworks: a nested directed graph that breaks down tasks into multiple LLM steps, with branching and recursion to enable agent-like decision-making. From there, you can layer on more advanced features like agents, RAG, task decomposition, and more. I’ve intentionally…
2025 · github.com
- 11SC
I created a tool that consolidates information from the following inputs: GitHub repository URL (e.g., https://github.com/jimmc414/onefilellm) arXiv abstract URL (e.g., https://arxiv.org/abs/2401.14295) Local folder path (e.g., C:\python\PipMyRide) Youtube video URL (e.g., https://www.youtube.com/watch?v=KZ_NlnmPQYk) Webpage URL (e.g., https://llm.datasette.io/en/stable/) It outputs the repo, web documentation, arXiv paper or YT transcript to a text file and the clipboard, displaying a token count. It also…
2024 · github.com
- 12HP
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
2023 · gethorizon.ai
- 13UL
Hi everyone, Just wanted to share a use case where local LLMs are genuinely helpful for daily workflows: file organization. I've been working on a C++ desktop app called AI File Sorter – it uses local LLMs via `llama.cpp` to help organize messy folders like `Downloads` or `Desktop`. Not sort files into folders solely based on extension or filename patterns, but based on what each file actually is supposed to do or does. Basically: what would normally take me a great deal of time for dragging and sorting can now be done in a few. It's cross-platform (Windows/macOS/Linux), and fully…
2025 · github.com
- 14PA
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
2024 · promptl.ai
- 15PR
Hi HN, While building RAG agents, I noticed a lot of token budget was wasted on formatting overhead (HTML tags, JSON structure, whitespace). Existing solutions felt too heavy (often requiring torch/transformers), so I wrote this lightweight, zero-dependency library to solve it. It includes strategies for context packing, PII redaction, and tool output compression. Benchmarks show it can save ~15% of tokens with negligible latency overhead (<0.5ms). Happy to answer any questions!
Dec 2025 · github.com
- 16AS
There are plenty of good tools for load testing such as JMeter, Minigun, and plenty more - but they all have their own set of options to learn. I built this one to support a simpler workflow where you just paste your curl request (or other cli command) after your `spam` config (literally just `spam -r 2 -- curl www.google.com). It's pretty barebones but I'm 100% going to use it in my day to day - figured I'd share it here if it helped anyone else!
2023 · github.com
- 17CA
Hi HN, I've been working with LLMs in production for a while both as a solo dev building apps for clients and working at an AI startup. The one thing that always was a pain was to pay OpenAI/Gemini/Anthropic a few dollars a month just for me to say "test" or have a CI runner validate some UI code. So I built this server called ChunkBack, that mocks the popular llm provider's functionality but allows you to type in a deterministic language: `SAY "cheese"` or `TOOLCALL "tool_name" {} "tool response"` I've had to work in some test environments and give good results for experimenting…
Nov 2025 · github.com
- 18IL
LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more. There are thousands of permutations to try. We need to be able to easily experiment with these…
2024 · palico.ai
- 19AI
Hi I am Jan, CTO @ Pathway. A use case we have been working on with LLMs is to let people know when an answer to their query changes due to revisions of source documents. Obviously, we want to avoid periodically re-computing all queries for the LLM. Why I think it’s cool? - We don’t spin in a loop to repeat with the LLM. - Alerts are LLM-deduplicated - no spamming users with typo fixes - And the best - our framework, Pathway takes care of handling the updates, the example looks nearly like a regular, static RAG chatbot. More context + GIF of how it works for Google Drive document alerts:…
2023 · github.com
- 20IB
After fine-tuning GPT for a personal project, I realized how tedious it is to write plain text in a massive JSON file. That's why I built this app for my own use, and I want to see if others could benefit from a tool like this as well ;)
2024 · finetuna-ui.com
- 21LP
2025 · codehooks.io
- 22AT
2025 · tella.tv
- 23LF
Hey HN, I built SWE-Kit, LLM toolkit (Function callable tools) which makes building agents specialised in coding like Devin very easy. I noticed a typical pattern while building local agents: creating & perfecting LLM tools to interact with system or codebase was the repeated and time-consuming. We created a layer that simplifies building agents that can interact with code, file system, git, shell and allows you to quickly solve for a wide variety of coding agent use cases. Aren’t there open coding agents already? Well, yes, but most folks would want to solve their specific use case like a…
2024 · swekit.dev
- 24IB
Hi HN, I'm pleased to share Promptspot, an open-source (Apache License 2.0) project that helps automate testing of large language model (LLM) prompts against an array of input data. Modern LLMs offer an enormous amount of leverage if you "teach the bot to fish" — i.e. simply prompt it with both a "system prompt" (which typically doesn't change often) and a dynamic input, which is often application state, search results, recent activity, user profile data, etc. Existing playgrounds and prompt management systems often lack the rigor and flexibility required for this dynamic approach — and as…
2023 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →