Marvin – build AI functions that use an LLM as a runtime
Hey HN! We're excited to share our new open-source project, Marvin. Marvin is a high-level library for building AI-powered software. We developed it to address the challenges of integrating LLMs into more traditional applications. One of the biggest issues is the fact that LLMs only deal with strings (and conversational strings at that), so using them to process structured data is especially difficult. Marvin introduces a new concept called AI Functions. These look and feel just like regular Python functions: you provide typed inputs, outputs, and docstrings. However, instead of relying on…
In plain words
Marvin is an open-source Python library for building AI-powered applications that use large language models as a runtime. It introduces AI Functions, which work like regular Python functions but execute through LLMs such as GPT-4 instead of traditional code. Users define typed inputs and outputs, and Marvin handles converting string-based LLM responses back into structured data types. This approach simplifies integrating language models into traditional applications that require processing structured data rather than conversational strings.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN! We're excited to share our new open-source project, Marvin. Marvin is a high-level library for building AI-powered software. We developed it to address the challenges of integrating LLMs into more traditional applications. One of the biggest issues is the fact that LLMs only deal with strings (and conversational strings at that), so using them to process structured data is especially difficult. Marvin introduces a new concept called AI Functions. These look and feel just like regular Python functions: you provide typed inputs, outputs, and docstrings. However, instead of relying on traditional source code, AI functions use LLMs like GPT-4 as a sort of “runtime” to generate outputs on-demand, based on the provided inputs and other details. The results are then parsed and converted back into native data types. This “functional prompt engineering” means you can seamlessly integrate AI functions with your existing codebase. You can chain them together with other functions to form sophisticated, AI-enabled pipelines. They’re particularly useful for tasks that are simple to describe yet challenging to code, such as entity extraction, semantic scraping, complex filtering, template-based data generation, and categorization. For example, you could extract terms from a contract as JSON, scrape websites for quotes that support an idea, or build a list of questions from a customer support request. All of these would yield structured data that you could immediately start to process. We initially created Marvin to tackle broad internal use cases in customer service and knowledge synthesis. AI Functions are just a piece of that, but have proven to be even more effective than we anticipated, and have quickly become one of our favorite features! We’re eager for you to try them out for yourself. We’d love to hear your thoughts, feedback, and any creative ways you could use Marvin in your own projects. Let’s discuss in the comments!
Does the same job
all alternatives →- M2Marvin 2.0 – a lightweight, multi-modal AI toolkit2024 · factsmachine.ai · ▲20
Hey HN! We just released Marvin 2.0. Marvin is an AI toolkit for developers who want to use LLMs with traditional software. We still see significant challenges integrating LLMs because of how difficult it is to get them to reliably accept and return structured data. Marvin consists of independent, functional tools that address this problem in a variety of ways. Marvin has always been focused on using LLMs to work with native Python datatypes and Pydantic models. In 2.0 we've expanded this significantly with dedicated APIs for the most common use cases we've seen over the last year:…
- HTHow to use LLMs to generate accurate SQL for real-world data2023 · github.com · ▲21
Hey HN, We are Zain and Ashish, founders of Vanna AI. We recently embarked on an experiment to see if large language models (specifically LLMs) could help in generating SQL queries for real-world datasets. We initially started this project as a web app but realized that it was most useful and had broadest applicability as a Python package since you can then incorporate it into an existing workflow (Jupyter notebook, Slackbot, etc). We've had some good success with customer datasets but we've generally heard a lot of skepticism so we decided to write a paper about the methodology we're using…
- HWHow we leapfrogged traditional vector based RAG with a 'language map'2024 · twitter.com · ▲162
TL;DR: Vector-based RAG performs poorly for many real-world applications like codebase chats, and you should consider 'language maps'. Part of our mission at Mutable.ai is to make it much easier for developers to build and understand software. One of the natural ways to do this is to create a codebase chat, that answer questions about your repo and help you build features. It might seem simple to plug in your codebase into a state-of-the-art LLM, but LLMs have two limitations that make human-level assistance with code difficult: 1. They currently have context windows that are too small to…
- VAVanna AI – Open-sourced text-to-SQL in Python2023 · vanna.ai · ▲33
Hey there HN! We've just open-sourced Vanna – a Python package that allows you to transform questions into SQL. We've leveraged LLMs to enable you to "ask" databases what you need, bypassing the need to "write" complex SQL. Quick Overview: - "Train" using DDL statements, documentation, or known correct SQL statements. - "Ask" questions in natural language and receive SQL, tables, and charts in return. - Open Source Flexibility: Swap storage mechanisms, customize LLMs, and choose your databases. - Local or Hosted: Operate everything locally or use our hosted version for free (including…
- MCMonkeyPatch – Cheap, fast and predictable LLM functions in Python2023 · github.com · ▲95
Hi HN, Jack here! I'm one of the creators of MonkeyPatch, an easy tool that helps you build LLM-powered functions and apps that get cheaper and faster the more you use them. For example, if you need to classify PDFs, extract product feedback from tweets, or auto-generate synthetic data, you can spin up an LLM-powered Python function in <5 minutes to power your application. Unlike existing LLM clients, these functions generate well-typed outputs with guardrails to mitigate unexpected behavior. After about 200-300 calls, these functions will begin to get cheaper and faster. We've seen 8-10x…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, March 2023
the whole month →
- BI
I'm a big fan of the BBC podcast In Our Time -- and (like most people) I've been playing with the OpenAI APIs. In Our Time has almost 1,000 episodes on everything from Cleopatra to the evolution of teeth to plasma physics, all still available, so it's my starting point to learn about most topics. But it's not well organised. So here are the episodes sorted by library code. It's fun to explore. Web scraping is usually pretty tedious, but I found that I could send the minimised HTML to GPT-3 and get (almost) perfect JSON back: the prompt includes the Typescript definition. At the same time I…
AI · 2023 · genmon.github.io