Ergonomically call LLM in bulk from CLI
Hi! I've found myself repeatedly writing little scripts to do bulk calls to LLMs for various tasks. For example, run some analysis on a large list of records. There are a few "gotchas" to doing this. For example, some service providers have rate limits, and some models will not reliably return JSON (if you're asking for it). So, I've written a command for this. What I've tried to do here is let the user break up prompts and configuration as they see fit. For example, you can have a prompt file which includes the API key, rate limit, settings, etc. all together, or break these up into…
What it does
In the maker’s words, at launch
Hi! I've found myself repeatedly writing little scripts to do bulk calls to LLMs for various tasks. For example, run some analysis on a large list of records. There are a few "gotchas" to doing this. For example, some service providers have rate limits, and some models will not reliably return JSON (if you're asking for it). So, I've written a command for this. What I've tried to do here is let the user break up prompts and configuration as they see fit. For example, you can have a prompt file which includes the API key, rate limit, settings, etc. all together, or break these up into multiple files, or keep some parts local, or override parameters. This solves the problem of sharing settings between activities, and keeping prompts in simple, committable files of narrow scope. I hope this can be of use to someone. Thanks for reading.
Does the same job
all alternatives →
- LBlambdaprompt – build, compose and call templated LLM prompts2022 · github.com · ▲9
For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…
- KCkey-carousel - Key rotation for LLM agentsMar 2026 · github.com · ▲5
I think in-process key management is the right abstraction for multi-key LLM setups. Not LiteLLM, not a Redis queue, not a custom load balancer. The failure modes are well-understood: a key gets rate-limited, you wait, you try the next one. Billing errors need a longer cooldown than rate limits. This is not a distributed systems problem — it's a state machine that fits in a library. The problem is everyone keeps solving it with infrastructure instead. Spin up LiteLLM, now you have a Python service to maintain. Reach for Redis, now you have a database for a problem that doesn't need one.…
- CLCompress Long LLM Prompt, Right in your web browser2025 · atjsh.github.io · ▲5
I tried to port LLMLingua-2's official Python implementation into TypeScript. For best performance, open the URL with a WebGPU enabled web browser. Learn More: https://github.com/atjsh/llmlingua-2-js
- EAExtrai – An open-source tool to fight LLM randomness in data extractionNov 2025 · github.com · ▲5
A few months ago I was working on a flight search engine that would include pet transport costs (I know a few by hearth but storing them and make the calculations in the UI would be nice) While I was collecting pet pricing from several airlines I strugled to extract data in a common format without hallucinated values. That's when I thought: What if I use multiple LLMs and take the most common response to improve accuracy? This idea became this new project. You provide your documents, an SQLModel schema, an LLM provider, plus what you'd like to extract and Extrai does the rest. Including…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, January 2025
the whole month →- IM
Hello! I'm Byran. I spent the past ~6 months engineering a laptop from scratch. It's fully open-source on GH at: https://github.com/Hello9999901/laptop
Dev tools · 2025 · byran.ee
- TITetris in a PDF▲1,289
I realized that the PDF engines of modern desktop browsers (PDFium and PDF.js) support JavaScript with enough I/O primitives to make a basic game like Tetris. It was a bit tricky to find a union of features that work in both engines, but in the end it turns out that showing/hiding annotation "fields" works well to make monochrome pixels, and keyboard input can be achieved by typing in a text input box. All in all it's quite janky but a nice reminder of how general purpose PDF scripting can be. The linked PDF is all ASCII so you can just open it in a text editor, or have a look at…
Life & fun · 2025 · th0mas.nl
Create lifelike, personalized AI avatars from text prompts
AI · 2025 · jogg.ai
