MonkeyPatch – Cheap, fast and predictable LLM functions in Python
Hi HN, Jack here! I'm one of the creators of MonkeyPatch, an easy tool that helps you build LLM-powered functions and apps that get cheaper and faster the more you use them. For example, if you need to classify PDFs, extract product feedback from tweets, or auto-generate synthetic data, you can spin up an LLM-powered Python function in <5 minutes to power your application. Unlike existing LLM clients, these functions generate well-typed outputs with guardrails to mitigate unexpected behavior. After about 200-300 calls, these functions will begin to get cheaper and faster. We've seen 8-10x…
MonkeyPatch – Cheap, fast and predictable LLM functions in Python launched on November 15, 2023 with 95 votes, #768 of 4,333 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
MonkeyPatch is a Python tool for building LLM-powered functions that become cheaper and faster with repeated use. Developers can create functions for tasks like PDF classification, data extraction, or synthetic data generation in under five minutes, with outputs that include type safety and guardrails. The tool uses progressive knowledge distillation to automatically fine-tune smaller models in the background, with users typically seeing significant cost and latency improvements after 200-300 function calls.
written from the facts on this page · September 2026
From the sources
Prompt engineering for developers. Contribute to Tanuki/tanuki.py development by creating an account on GitHub.
In the maker’s words, at launch
Hi HN, Jack here! I'm one of the creators of MonkeyPatch, an easy tool that helps you build LLM-powered functions and apps that get cheaper and faster the more you use them. For example, if you need to classify PDFs, extract product feedback from tweets, or auto-generate synthetic data, you can spin up an LLM-powered…
Does a similar job
all alternatives →
LLMs can generate valid JSON 100% of the time2023 · github.com · ▲854Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
Magentic – Use LLMs as simple Python functions2023 · github.com · ▲283This is a Python package that allows you to write function signatures to define LLM queries. This makes it easy to mix regular code with calls to LLMs, which enables you to use the LLM for its creativity and reasoning while also enforcing structure/logic as necessary. LLM output is parsed for you according to the return type annotation of the function, including complex return types such as streaming an array of structured objects. I built this to show that we can think about using LLMs more fluidly than just chains and chats, i.e. more interchangeably with regular code, and to make it…
Magic Loops – Combine LLMs and code to create simple automations2023 · magicloops.dev · ▲248Howdy! We built this as an experiment in personal-programming, combining the best of LLMs and code to help automate tasks around you. I personally use it to track the tides and get notified when certain conditions are met, something that pure LLMs had trouble dealing with and pure code was often too brittle for. We created it after getting frustrated with the inability of LLMs to deal with numbers and the various hoops we had to jump through to make ChatGPT output repeatable. At the core, Magic Loops are just a series of "blocks" (JSON) that can be triggered with different inputs (email,…
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
Marvin – build AI functions that use an LLM as a runtime2023 · github.com · ▲127Hey HN! We're excited to share our new open-source project, Marvin. Marvin is a high-level library for building AI-powered software. We developed it to address the challenges of integrating LLMs into more traditional applications. One of the biggest issues is the fact that LLMs only deal with strings (and conversational strings at that), so using them to process structured data is especially difficult. Marvin introduces a new concept called AI Functions. These look and feel just like regular Python functions: you provide typed inputs, outputs, and docstrings. However, instead of relying on…
Modshim – A new alternative to monkey-patching in Python2025 · github.com · ▲109I've invented a new alternative to forking / vendoring / monkey-patching packages in Python. It's a bit like OverlayFS for Python modules - it allows you write modifications for a target module (lower) in a new module (upper), and have these combined in a new virtual module (mount). It works by rewriting imports using AST transformations, then running both the lower and upper module's code in the new Python module. This prevents polluting the global namespace when monkey-patching, and means if you want to make changes to a third-party package, you don't have to take on the…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, November 2023
the whole month →
Discover & book top creators to promote your product
Growth · 2023 · passionfroot.me
SpeedVitals▲829Hi HN, I’m Kashish, one of the co-founders of SpeedVitals, a platform for testing & monitoring website performance with a major emphasis on Core Web Vitals. Until now, our platform has only focused on Synthetic/Lab testing, such as Lighthouse tests and TTFB tests. But now, we have developed a Real-User Monitoring platform that doubles up as a Web Analytics tool. One major feature we have that most RUM tools lack is Performance Audits. Just like Lighthouse audits work on Lab test data, our audits work across all the real-user visits and gives suggestions on what task to prioritize first.…
Dev tools · 2023 · speedvitals.com