Prompts as WASM Programs
AICI is a proposed common interface between LLM inference engines (llama.cpp, vLLM, HF Transformers, etc.) and "controllers" - programs that can constrain the LLM output according to regexp, grammar, or custom logic, as well as control the generation process (forking, backtracking, etc.). AICI is based on Wasm, and is designed to be fast (runs on CPU while GPU is busy), secure (can run in multi-tenant cloud deployments), and flexible (allow libraries like Guidance, LMQL, Outlines, etc. to work on top of it). We (Microsoft Research) have released it recently, and would love feedback on the…
Prompts as WASM Programs launched on March 11, 2024 with 209 votes, #228 of 4,270 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
AICI is a standardized interface that connects large language model inference engines like llama.cpp and vLLM with controller programs written in WebAssembly. These controllers constrain LLM output using regular expressions, grammars, or custom logic, and can manage the generation process through forking and backtracking. Developed by Microsoft Research, AICI is designed to execute efficiently on CPUs while GPUs handle inference, operate securely in shared cloud environments, and enable libraries like Guidance and Outlines to build on top of it.
written from the facts on this page · September 2026
From the sources
AICI: Prompts as (Wasm) Programs. Contribute to microsoft/aici development by creating an account on GitHub.
In the maker’s words, at launch
AICI is a proposed common interface between LLM inference engines (llama.cpp, vLLM, HF Transformers, etc.) and "controllers" - programs that can constrain the LLM output according to regexp, grammar, or custom logic, as well as control the generation process (forking, backtracking, etc.). AICI is based on Wasm, and…
Does a similar job
all alternatives →
Marvin – build AI functions that use an LLM as a runtime2023 · github.com · ▲127Hey HN! We're excited to share our new open-source project, Marvin. Marvin is a high-level library for building AI-powered software. We developed it to address the challenges of integrating LLMs into more traditional applications. One of the biggest issues is the fact that LLMs only deal with strings (and conversational strings at that), so using them to process structured data is especially difficult. Marvin introduces a new concept called AI Functions. These look and feel just like regular Python functions: you provide typed inputs, outputs, and docstrings. However, instead of relying on…
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)2023 · github.com · ▲45G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
Improve LLM Performance by Maximizing Iterative Development2024 · github.com · ▲104I have been working in AI space for a while now, first at FAANG with ML since 2021, then with LLM in start-ups since early 2023. I think LLM Application development is extremely iterative, more so than any other types of development. This is because to improve an LLM application performance (accuracy, hallucinations, latency, cost), you need to try various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, different agent architecture, and more. There are thousands of possible combinations and you need a process…
Autofix Bot – Hybrid static analysis and AI code review agentDec 2025 · deepsource.com · ▲37Hi there, HN! We’re Jai and Sanket from DeepSource (YC W20), and today we’re launching Autofix Bot, a hybrid static analysis + AI agent purpose-built for in-the-loop use with AI coding agents. AI coding agents have made code generation nearly free, and they’ve shifted the bottleneck to code review. Static-only analysis with a fixed set of checkers isn’t enough. LLM-only review has several limitations: non-deterministic across runs, low recall on security issues, expensive at scale, and a tendency to get ‘distracted’. We spent the last 6 years building a deterministic, static-analysis-only…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, March 2024
the whole month →
🌐 The Internet Computer! Free, Open-Source, and Self-Hostable. - HeyPuter/puter
Dev tools · 2024 · github.com
Microlaunch▲1,115Launch and get feedback on both the idea and product
Dev tools · 2024 · microlaunch.net
