Debugging LLM Failures Like "9.11 > 9.9" via Interpretability

Hey HN, I just built an experimental VSCode extension called LLM Debugger. It’s a proof-of-concept that lets a large language model take charge of debugging. Instead of only looking at the static code, the LLM also gets to see the live runtime state—actual variable values, function calls, branch decisions, and more. The idea is to give it enough context to help diagnose issues faster and even generate synthetic data from running programs. Here’s what it does: * Active Debugging: It integrates with Node.js debug sessions to gather runtime info (like variable states and stack traces). *…

A panel about failure at Code Conference 2016

An open source exit intent detection library

Debug failed LLM calls with root-cause diagnoses

Hi Hacker News! We’re Vadim and Chris from Highlight.io [1]. We do web app monitoring and are working on using LLMs/embeddings to add new functionality to our error monitoring product. Given that there’s a lot of founders/engineers using LLMs in their products, we figured we’d share how we built the new functionality, their impact on our workflows, and how you can try it out. Our goal was to build two features: (1) tagging errors (e.g. deeming an error as “authentication error” or a “database error”); and (2) grouping similar errors together (e.g. two errors that have a different…