70 days, 800 stars. If AI bugs are not random but math inevitable?
hi all. i’ve been shipping a small open project that tries to answer that question with evidence, not vibes. in 70 days it reached \~800 stars. the core claim is simple: many AI failures are not noise. they repeat because the geometry and ordering underneath are stable. if so, we should be able to name each failure mode, set acceptance targets, and stop shipping the same bug twice. ### what it is * a compact Problem Map of 16 reproducible failure modes in RAG and agents. * each item has a minimal fix and measurable gates. examples: * Semantic ≠ Embedding: metric and normalization mismatch.…
What it does
In the maker’s words, at launch
hi all. i’ve been shipping a small open project that tries to answer that question with evidence, not vibes. in 70 days it reached \~800 stars. the core claim is simple: many AI failures are not noise. they repeat because the geometry and ordering underneath are stable. if so, we should be able to name each failure mode, set acceptance targets, and stop shipping the same bug twice. ### what it is * a compact Problem Map of 16 reproducible failure modes in RAG and agents. * each item has a minimal fix and measurable gates. examples: * Semantic ≠ Embedding: metric and normalization mismatch. accept if coverage of target section ≥ 0.70 and deltaS(question, retrieved) ≤ 0.45 across 3 paraphrases. * Logic Collapse & Recovery: synthesis runs on thin evidence. require a bridge step before answering. * Memory Breaks Across Sessions: new chat loses context. use metadata trace then reattach. * Bootstrap Ordering / Pre-deploy Collapse: shipped an empty or mixed index. block deploy until ingest counts and retrieval smoke tests pass. * MIT licensed. no SDK, no telemetry, no infra change. ### why i believe this is true * repeated A/B/C runs across mainstream models show the same patterns returning. * small changes in metric, normalization, or chunk contract flip outcomes in a predictable way. * when you enforce simple gates, detours drop and chains stabilize across paraphrases. ### quick falsification you can run 1. pick any non-toy question where your system struggles. 2. run it three ways: retriever only, retriever+rerank, and with a bridge step that refuses to answer on thin evidence. 3. measure: coverage of the target span, deltaS(question, retrieved), citations per atomic claim, answer stability across 3 paraphrases. 4. if coverage is low and only looks good after rerank, you are likely in Semantic ≠ Embedding. if coverage is ok but prose still drifts, it is Logic Collapse. if a fresh chat forgets prior context, it is Memory Breaks. these are all predictable, not random. ### what i’m asking from HN * try to break it. if you have a counterexample where the gates do not stabilize the chain, i want to see it. * if you maintain a vector store, agent framework, or eval suite, tell me where this framing fails in the real world. * if the map helps you ship fewer regressions, say which item saved you time so we can harden that fix. happy to answer pointed questions. if this is wrong, i’d like to know exactly where the math breaks. if it is roughly right, maybe we can stop treating these bugs as mysterious and start treating them like unit failures with thresholds.
Does the same job
all alternatives →- SFSubtle Failure Modes I Keep Seeing in Production‑Grade AI Systems2025 · github.com · ▲6
Hi HN, Over the past two years I’ve built and debugged a fair number of production pipelines—mainly retrieval‑augmented generation stacks, agent frameworks, and multi‑step reasoning services. A pattern emerged: most incidents weren’t outright crashes, but silent structural faults that slowly compromised relevance, accuracy, or stability. I began logging every recurring fault in a shared notebook. Colleagues started using the list for post‑mortems, so I turned it into a small public reference: 16 distinct failure modes (semantic drift after chunking, embedding/meaning mismatches,…
- SOState of AI Agents 2024 – 184B tokens · 786M runs analyzed2024 · langbase.com · ▲6
hello everyone, my first post! AA here, founder of ⌘ Langbase.com — we are a developer platform for building and scaling serverless AI memory agents. I know surveys can be boring, but this one’s different—it’s interactive! That's very much intentional. My team and I have been up for the last 21 hours putting together this report. This was a looot of work, so I hope y'all like it. Introducing … State of AI Agents 2024 report On Langbase, we processed 184 billion tokens and handled 786 million AI agent runs from 36K developers. From all that data plus insights from 3.4K builders who filled out…
- SASup AI, a confidence-weighted ensemble (52.15% on Humanity's Last Exam)Mar 2026 · sup.ai · ▲26
Hi HN. I'm Ken, a 20-year-old Stanford CS student. I built Sup AI. I started working on this because no single AI model is right all the time, but their errors don’t strongly correlate. In other words, models often make unique mistakes relative to other models. So I run multiple models in parallel and synthesize the outputs by weighting segments based on confidence. Low entropy in the output token probability distributions correlates with accuracy. High entropy is often where hallucinations begin. My dad Scott (AI Research Scientist at TRI) is my research partner on this. He sends me papers…
- W1Watch 14-Byte AI "brains" attempt to solve a 2D maze (Its hard)Jul 2026 · con-dog.github.io · ▲25
Hey HackerNews, I built this project over the last few weeks as a palette cleanser from a failed game launch. I wanted to learn a bit about AI/Neural-Networks and naively thought I could build a tiny maze-solving AI in a weekend with a 100% solve rate. Well - I couldn't, but I got pretty close. 14 Bytes total model size, and a 96.5% solve rate on unseen mazes. Trained across 46 phases experimenting with different ideas to improve the model (better performance, smaller size). Its quite fun to watch the model attempt to solve the maze, when they fail its usually due to getting stuck in a…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, August 2025
the whole month →
- IS
I built the world's most impractical 1000-pixel display and anyone in the world can draw on it. It draws a single pixel at a time and takes 30-60 minutes to complete a single image. Anyone can participate in the project by voting for the next image to be drawn, and submitting images. https://kilopx.com/
Work · 2025 · benholmen.com
- KT
Kitten TTS is an open-source series of tiny and expressive text-to-speech models for on-device applications. We are excited to launch a preview of our smallest model, which is less than 25 MB. This model has 15M parameters. This release supports English text-to-speech applications in eight voices: four male and four female. The model is quantized to int8 + fp16, and it uses onnx for runtime. The model is designed to run literally anywhere eg. raspberry pi, low-end smartphones, wearables, browsers etc. No GPU required! We're releasing this to give early users a sense of the latency and voices…
Dev tools · 2025 · github.com
- IW
I was wondering how I can arrange objects along a spherical helix path, and read some articles on it. I ended up learning about parametric equations again, and make this visualization to document what I learned: https://visualrambling.space/moving-objects-in-3d/ feel free to visit and let me know what you think!
Life & fun · 2025 · visualrambling.space
- TC
For HTML Day 2025 [1], I made a web service that displays the current sky at your approximate location as a CSS gradient. Colours are simulated on-demand using atmospheric absorption and scattering coefficients. Updates every minute, without the use of client-side JavaScript. Source code and additional information is available on GitHub: https://github.com/dnlzro/horizon [1] https://html.energy/html-day/2025/index.html
Dev tools · 2025 · sky.dlazaro.ca