Axilla – Open-source TypeScript framework for LLM apps
Hi HN, we are Nick and Ben, creators of Axilla - an open source TypeScript framework to develop LLM applications. It’s in the early stages but you can use it today: we’ve already published 2 modules and have more coming soon. Ben and I met while working at Cruise on the ML platform for self-driving cars. We spent many years there and learned the hard way that shipping AI is not quite the same as shipping regular code. There are many parts of the ML lifecycle, e.g., mining, processing, and labeling data and training, evaluating, and deploying models. Although none of them are rocket science,…
In plain words
Axilla is an open-source TypeScript framework for building applications with large language models. Created by former Cruise engineers, it provides tools for developing LLM apps and currently includes two published modules with more planned. The framework addresses the challenge of integrating multiple stages of the ML lifecycle—from data processing to model deployment—which the creators found to be a major source of inefficiency when shipping AI applications.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN, we are Nick and Ben, creators of Axilla - an open source TypeScript framework to develop LLM applications. It’s in the early stages but you can use it today: we’ve already published 2 modules and have more coming soon. Ben and I met while working at Cruise on the ML platform for self-driving cars. We spent many years there and learned the hard way that shipping AI is not quite the same as shipping regular code. There are many parts of the ML lifecycle, e.g., mining, processing, and labeling data and training, evaluating, and deploying models. Although none of them are rocket science, most of the inefficiencies tend to come from integrating them together. At Cruise, we built an integrated framework that accelerated the speed of shipping models to the car by 80%. With the explosion of generative AI, we are seeing software teams building applications and features with the same inefficiencies we experienced at Cruise. This got us excited about building an opinionated, end-to-end platform. We started building in Python but quickly noticed that most of the teams we talked to weren’t using Python, but instead building in TypeScript. This is because most teams are not training their own models, but rather using foundational ones served by third parties over HTTP, like openAI, anthropic or even OSS ones from hugging face. Because of this, we’ve decided to build Axilla as a TypeScript first library. Our goal is to build a modular framework that can be adopted incrementally yet benefits from full integration. For example, the production responses coming from the LLM should be able to be sent — with all necessary metadata — to the eval module or the labeling tooling. So far, we’ve shipped 2 modules, that are available to use today on npm: * *axgen*: focused on RAG type workflows. Useful if you want to ingest data, get the embeddings, store it in a vector store and then do similarity search retrieval. It’s how you give LLMs memory or more context about private data sources. * *axeval*: a lightweight evaluation library, that feels like jest (so, like unit tests). In our experience, evaluation should be really easy to setup, to encourage continuous quality monitoring, and slowly build ground truth datasets of edge cases that can be used for regression testing, and fine-tuning. We are working on a serving module and a data processing one next and would love to hear what functionality you need us to prioritize! We built an open-source demo UI for you to discover the framework more: https://github.com/axilla-io/demo-ui And here's a video of Nicholas walking through the UI that gives an idea of what axgen can do: https://www.loom.com/share/458f9b6679b740f0a5c78a33fffee3dc We’d love to hear your feedback on the framework, you can let us know here, create an issue on the GitHub repo or send me an email at [email protected] And of course, contributions welcome!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, August 2023
the whole month →
Lottielab▲871Create and ship lottie animations to sites and apps faster
Dev tools · 2023 · lottielab.com
- LC
Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
AI · 2023 · github.com