Alternatives
Products that do what VLLM with JSON Guided Generation does
Our project, Outlines, now offers guided/constrained generation (e.g. according to a JSON schema) via the VLLM library. My colleague, Rémi, created some patches that allow one to pass vLLM a JSON schema along with the prompt, which dramatically simplifies deployment of JSON-guided generation. He also added a new `serve` interface that puts it all together and makes serving such models a 2-3 line process. Check it out and tell us what you think!
- 1LC
Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
2023 · github.com
- 2AJ
Hey HN, we’re building an open specification that lets agents discover and invoke APIs with natural language, built on the OpenAPI standard. agents.json clearly defines the contract between LLMs and API as a standard that's open, observable, and replicable. Here’s a walkthrough of how it works: https://youtu.be/kby2Wdt2Dtk?si=59xGCDy48Zzwr7ND. There’s 2 parts to this: 1. An agents.json file describes how to link API calls together into outcome-based tools for LLMs. This file sits alongside an OpenAPI file. 2. The agents.json SDK loads agents.json files as tools for an LLM that…
2025 · github.com
- 3SO
Built a tool for transforming unstructured data into structured outputs using language models (with 100% adherence). If you're facing problems getting GPT to adhere to a schema (JSON, XML, etc.) or regex, need to bulk process some unstructured data, or generate synthetic data, check it out. We run our own tuned model (you can self-host if you want), so, we're able to have incredibly fine grained control over text generation. Repository: https://github.com/automorphic-ai/trex Playground: https://automorphic.ai/playground
2023 · automorphic.ai
- 4
- 5KO
We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty. What Klarity does: - Real-time analysis of model uncertainty during generation - Dual analysis combining log probabilities and semantic understanding - Structured JSON output with actionable insights - Fully self-hostable with customizable analysis models The tool works by analyzing each step of text generation and returns a structured JSON: - uncertainty_points: array of {step, entropy,…
2025 · github.com
- 6OE
Built an open JSON Schema for defining AI agent teams. Multi-agent systems are becoming a real deployment pattern — not single assistants, but teams with roles, handoffs, and human checkpoints. But there's no shared way to define one that travels across frameworks. Every implementation is scattered, locked to whichever tool you picked first. Built the schema to fix that. The schema lives at schema.openenvelope.org and is registered in SchemaStore, so if you drop a .envelope.json file in VS Code you get autocomplete and validation without installing anything. It's also on npm as…
May 2026 · openenvelope.org
- 7JA
2017 · maxleiter.github.io
- 8LL
May 2026 · github.com
- 9AT
Hey HN! Erik here from banana.dev We’ve trained a small(ish) language model on structured extraction, and today we’re launching a playground for it at https://anythingtojson.com. Give it a try! This model continues our work on structured generation, following last week’s launch of Fructose[1], a python client for strongly-typed LLM responses. There seem to be two distinct halves of the problem intended to be solved by Fructose and structured generation: 1. the reasoning ability of the model, such as performing chain of thought, creative acts, and natural language tasks. In a way,…
2024 · anythingtojson.com
- 10UL
Recently featured in a LangChain blog https://blog.langchain.dev/empowering-development-with-flowt... , use LLMs to construct an API first runnable workflow with an IDE experience.
2024 · github.com
- 11BP
2024 · instill.tech
- 12AT
Hello all To preface this is just something I've been making as a learning exercise, so all feedback is appreciated. This is a tool that converts JSON schemas into TypeScript utility classes for use in Deno. Automatic Type Generation: Typescript interfaces for the compressed and uncompressed versions of your data. Compression & Decompression: Compress and decompress your data. Validation: Built-in data validation using Ajv ensures your data adheres to the schema. Reusability: Once generated, the utility classes can be used in other Deno projects. It currently only supports a subset of JSON…
2023 · github.com
- 13CL
2018 · github.com
- 14JI
Hi HN, I built a code generator plugin for IntelliJ that uses LLMs to create repetitive Java code like implementations, tests, and fixtures — based on custom natural-language patterns and annotation-based references. Most tools like Copilot or Cursor aim to be general, but fail to produce code that actually fits a project structure or passes tests. So I made something more explicit: define patterns + reference scope, and generate code consistently. In this demo, 400 lines of Java were generated in 20 seconds — and all tests passed: https://www.youtube.com/watch?v=ReBCXKOpW3M…
2025 · github.com
- 15
- 16GG
llama.cpp added context-free grammar guided generation functionality. It requires passing a file in a derivative of BNF notation, which gets messy very quickly for things like JSON. To improve the experience, we built a small compiler from TypeScript interfaces to the grammar file format and have it hosted in a little browser app. See more in discussion at https://github.com/ggerganov/llama.cpp/discussions/2494
2023 · grammar.intrinsiclabs.ai
- 17OS
2023 · github.com
- 18OD
The Problem "Vibing" with LLMs is often too shallow for complex logic, while writing full specifications is cognitively expensive and slow. We need a middle ground that mimics how human programmers gather context—scanning structure before diving into details. The Solution: Outline Driven Development (ODD) I've built a "batteries-included" kit for Gemini/Claude/Codex that uses AST analysis to understand code structure rather than just raw text. This relies on a hyper-optimized Rust toolchain (`ast-grep`, `ripgrep`, `jj`, etc.) to feed precise, structural context to the agent. 1. The…
Nov 2025 · github.com
- 19KL
LLM agents often place raw JSON tool outputs directly in the prompt. After a few tool calls, earlier results get compacted or truncated and answers become incorrect or inconsistent. I built Sift, a drop-in MCP gateway that stores tool outputs as local artifacts (filesystem blobs indexed in SQLite) and returns an `artifact_id` plus compact schema hints when responses are large or paginated. Instead of reasoning over full JSON in the prompt, the model runs a small Python query: def run(data, schema, params): return max(data, key=lambda x: x["magnitude"])["place"] Query code runs in a…
Mar 2026 · github.com
- 20WA
We are pleased to launch a public preview of an open source JSON-LD database which combines the simplicity of a document db with the analytical power of semantic graph. We hope it is approachable for most any developer while having the capabilities, when needed, for data interoperability, embedded cell-level security (ReBAC style), rich shape/schema restrictions, data provenance (time travel), and fact inferencing. Why did our team at Fluree build this? Data is increasingly critical for great decision making, AI, and more. The way we typically store and manage data, mainly as an…
2023 · flur.ee
- 21TF
Hello all! Very happy to share this toolkit that allows you to fine-tune your choice of open-source LLMs on your data! The toolkit also allows you to run ablation studies across LLMs, prompt designs, training configurations, and can ingest different data files -- all through just ONE YAML file! After fine-tuning, you can also run a bunch of tests to ensure that the fine-tuned LLM behaves as expected, enabling faster time-to-production! Why this toolkit? Why now? While closed-source LLMs have become popular for chat-based applications, enterprises are considering a shift to self-hosted SLMs…
2024 · github.com
- 22HP
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
2023 · gethorizon.ai
- 23
Generate one JSON Schema from multiple samples
Jun 2026 · jsondevtools.org
- 24ZD
Hey HN! We just released a new library for building LLM-powered applications: @axflow/models. It is part of a larger suite of libraries we're developing for TypeScript developers working with generative AI. This library provides the simplest APIs for 1) invoking the most popular LLM and embedding models (openai, anthropic, cohere, huggingface, etc.) 2) streaming LLM responses to clients, including augmenting the streams with additional arbitrary data and 3) building client-side applications with React hooks. @axflow/models has zero dependencies and is built using only the…
2023 · docs.axflow.dev
Ranked by how close each launch is in meaning, then by votes. Refine with a description →