ChainFactory – Run Structured LLM Inference with Easy Parallelism

Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…

I've seen a lot of comments about how complex frameworks like LangChain can be. Over the holidays, I wanted to see how minimal an LLM framework could get if we stripped away everything non-essential. The result is an LLM framework in just 100 lines of code. These 100 lines capture what I see as the core abstraction of most LLM frameworks: a nested directed graph that breaks down tasks into multiple LLM steps, with branching and recursion to enable agent-like decision-making. From there, you can layer on more advanced features like agents, RAG, task decomposition, and more. I’ve intentionally…

Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…

For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…

Here is a production-first Keras-inspired LM framework, built with the advice of François Chollet (ex-Google, creator of Keras and ARC-AGI), our technical advisor. This system have already been deployed in production with our clients (which is why we have already every LLMOps practice implemented). It is also compatible with Jupyter and Marimo to integrate seamlessly in you Data Scientists workflows. You can try the code examples online on HF space and you can find more information in the documentation and FAQ. If you have any feedback for us don't hesitate to join our discord! More releases…

Hi, We are building an open-source framework for loading and structuring LLM context to create accurate and explainable LLM answers using knowledge graphs and vector stores. We built the tool with four main concepts in mind: 1. Loader -> uses dlt in the backend to load and structure the data 2. Cognify step -> creates a graph with summaries, labels and factoids that are interconnected across the documents and stored as a representation in the vector store 3. Optimizer -> Uses DSPy to optimize LLM queries, and we plan to extend it to most of the knobs we can turn, like chunking etc. 4. Search…