Alternatives
Products that do what Shim.so does
JSON repair API for broken LLM outputs. Sub-50ms streaming.
- 1LC
Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
2023 · github.com
- 2RL
We've been building data pipelines that scrape websites and extract structured data for a while now. If you've done this, you know the drill: you write CSS selectors, the site changes its layout, everything breaks at 2am, and you spend your morning rewriting parsers. LLMs seemed like the obvious fix — just throw the HTML at GPT and ask for JSON. Except in practice, it's more painful than that: - Raw HTML is full of nav bars, footers, and tracking junk that eats your token budget. A typical product page is 80% noise. - LLMs return malformed JSON more often than you'd expect, especially with…
Mar 2026 · github.com
- 3LA
Almost exactly 1 year ago, I submitted something to HN about using Llama2 (which had just come out) to improve the output of Tesseract OCR by correcting obvious OCR errors [0]. That was exciting at the time because OpenAI's API calls were still quite expensive for GPT4, and the cost of running it on a book-length PDF would just be prohibitive. In contrast, you could run Llama2 locally on a machine with just a CPU, and it would be extremely slow, but "free" if you had a spare machine lying around. Well, it's amazing how things have changed since then. Not only have models gotten a lot better,…
2024 · github.com
- 4
- 5
RAG-ready web scraping that cuts your LLM token costs
Apr 2026 · geekflare.com
- 6SO
Built a tool for transforming unstructured data into structured outputs using language models (with 100% adherence). If you're facing problems getting GPT to adhere to a schema (JSON, XML, etc.) or regex, need to bulk process some unstructured data, or generate synthetic data, check it out. We run our own tuned model (you can self-host if you want), so, we're able to have incredibly fine grained control over text generation. Repository: https://github.com/automorphic-ai/trex Playground: https://automorphic.ai/playground
2023 · automorphic.ai
- 7AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 8
- 9RL
While working with LLMs for structured web data extraction, we saw issues with invalid JSON and broken links in the output. This led me to build a library focused on robust extraction and enrichment: - Clean HTML conversion: transforms HTML into LLM-friendly markdown with an option to extract just the main content - LLM structured output: Uses Gemini 2.5 flash or GPT-4o mini to balance accuracy and cost. Can also also use custom prompt - JSON sanitization: If the LLM structured output fails or doesn't fully match your schema, a sanitization process attempts to recover and fix the data,…
2025 · github.com
- 10GJ
Hey HN, I've been using GPT a lot lately in some side projects around data generation and benchmarking. During the course of prompt tuning I ended up with a pretty complicated request: the value that I was looking for, an explanation, a criticism, etc. JSON was the most natural output format for this but results would often be broken, have wrong types, or contain missing fields. There's been some positive movement in this space, like with jsonformer (https://github.com/1rgs/jsonformer) the other day. But nothing that was plug and play with GPT. This library consolidates…
2023 · github.com
- 11TC
Hi HN, I spent my easter weekend stuck in the house with COVID and I decided to play with llama.cpp [1] and fauxpilot [2] to see if I could get LLM code assist working on pure CPU. As a proof of concept I'd say I've proven that it's possible. However there's still a lot to do. The auto complete is quite slow at the moment. PRs welcome. [1] https://github.com/ggerganov/llama.cpp [2] https://github.com/fauxpilot/fauxpilot
2023 · github.com
- 12AN
When building workflows that rely on LLMs, we commonly use structured output for programmatic use cases like converting an invoice into rows or meeting transcripts into tickets or even complex PDFs into database entries. The model may return the schema you want, but with hallucinated values like `invoice_date` being off by 2 months or the transcript array ordered wrongly. The JSON is valid, but the values are not. Structured output today is a big part of using LLMs, especially when building deterministic workflows. Current structured output benchmarks (e.g., JSONSchemaBench) only validate…
Apr 2026 · interfaze.ai
- 13
- 14FT
I wrote a small local tool to transcribe audio notes (Whisper/Parakeet). Code: https://github.com/bilawalriaz/lazy-notes I wanted to process raw transcripts locally without OpenRouter. Llama 3.2 3B with a prompt was decent but incomplete, so I tried SFT. I fine-tuned Llama 3.2 3B to clean/analyze dictation and emit structured JSON (title, tags, entities, dates, actions). Data: 13 real memos → Kimi K2 gold JSON → ~40k synthetic + gold; keys canonicalized. Chutes.ai (5k req/day). Training: RTX 4090 24GB, ~4h, LoRA (r=128, α=128, dropout=0.05), max seq 2048,…
2025 · bilawal.net
- 15IM
2024 · prakhar897.github.io
- 16IJ
Built this for streaming AI tool calls. LLMs stream function arguments as JSON character-by-character. Most parsers reparse from scratch each time - O(n²) behavior that causes UI lag. This maintains parsing state, processing only new characters. True O(n) performance that stays imperceptible throughout the entire response. Ruby gem, MIT licensed. Would love feedback.
Oct 2025 · aha.io
- 17LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com
- 18KL
LLM agents often place raw JSON tool outputs directly in the prompt. After a few tool calls, earlier results get compacted or truncated and answers become incorrect or inconsistent. I built Sift, a drop-in MCP gateway that stores tool outputs as local artifacts (filesystem blobs indexed in SQLite) and returns an `artifact_id` plus compact schema hints when responses are large or paginated. Instead of reasoning over full JSON in the prompt, the model runs a small Python query: def run(data, schema, params): return max(data, key=lambda x: x["magnitude"])["place"] Query code runs in a…
Mar 2026 · github.com
- 19IB
Hey HN -- I'm a solo dev. Built this because I got tired of AI crawlers reading my HTML in plain text while robots.txt did nothing. The core trick: shuffle characters and words in your HTML using a seed, then use CSS (flexbox order, direction: rtl, unicode-bidi) to put them back visually. Browser renders perfectly. textContent returns garbage. On top of that: email/phone RTL obfuscation with decoy characters, AI honeypots that inject prompt instructions into LLM scrapers, clipboard interception, canvas-based image rendering (no img src in DOM), robots.txt blocking 30+ AI crawlers, and…
Mar 2026 · obscrd.dev
- 20
- 21LS
Hi HN! Stefan here from superglue and today I’d like to share a new benchmark we’ve just open sourced: an Agent-API Benchmark, in which we test how well LLMs handle APIs. We gave LLMs API documentation and asked them to write code that makes actual API calls. Things like "create a Stripe customer" or "send a Slack message". We're not testing if they can use SDKs; we're testing if they can write raw HTTP requests (with proper auth, headers, body formatting) that actually work when executed against real API endpoints and can extract relevant information from that response. tl:dr: LLMs suck at…
2025 · github.com
- 22LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com
- 23AL
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →