Herd – A Go sidecar to stop stateful processes Puppeteer/LLMs from OOM
Hey HN. I'm an engineering student at Waterloo building stateful AI agents, and I kept hitting the same wall: whenever my Python scripts crashed or dropped a connection, the underlying Puppeteer or Ollama processes would just sit there orphaned, eating RAM until the node OOM-killed itself. Standard load balancers break sticky sessions, and passive HTTP timeouts are too slow for cleanup. I couldn't find a good local process pool that actually cleaned up dead stateful sessions reliably, so I built Herd in Go. It uses a persistent stream (gRPC/Unix sockets) strictly as a dead-man's switch.…
What it does
In the maker’s words, at launch
Hey HN. I'm an engineering student at Waterloo building stateful AI agents, and I kept hitting the same wall: whenever my Python scripts crashed or dropped a connection, the underlying Puppeteer or Ollama processes would just sit there orphaned, eating RAM until the node OOM-killed itself. Standard load balancers break sticky sessions, and passive HTTP timeouts are too slow for cleanup. I couldn't find a good local process pool that actually cleaned up dead stateful sessions reliably, so I built Herd in Go. It uses a persistent stream (gRPC/Unix sockets) strictly as a dead-man's switch. If your client script dies, the stream breaks. Herd registers the EOF and instantly fires a SIGKILL to the worker process (relying on Pdeathsig on Linux). For the actual heavy data, you just blast HTTP traffic through Herd's internal proxy, which routes it directly to the active process port. My actual goal is to turn this into a multi-node distributed mesh with a Redis registry, where a client can drop off and an edge gateway routes them back to the exact pod holding their stateful memory. But I know building a distributed mesh on top of a leaky local engine is a death sentence. The single-node cleanup has to be flawless first. I'd love for you guys to roast the architecture. Specifically: is relying on Pdeathsig actually robust enough for a local dead-man's switch in production, or am I being naive and need to just bite the bullet and wrap everything in cgroups & microvms right now? Repo link: https://github.com/herd-core/herd
Does the same job
all alternatives →
- SVStatewright – Visual state machines that make AI agents reliableMay 2026 · github.com · ▲126
Agentic problem solving in its current state is very brittle. I fell in love with it, but it creates as many problems as it solves. I'm Ben Cochran, I spent 20+ years in the trenches with full-stack Engineering, DevOps, high performance computing & ML with stints at NVIDIA, AMD and various other organizations most recently as a Distinguished Engineer. For agents to work reliably you either need massive parameter counts or massive context windows to keep the solution spaces workable. Most people are brute forcing reliability with bigger models and longer prompts. What if I made the problem…
- OAOodle.ai – $10 per million agent tracesJul 2026 · oodle.ai · ▲31
Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…
- GAGoru, an experimental, Go-inspired concurrency library for Ruby2023 · github.com · ▲6
Hey folks, wanted to show this off and get feedback. Still early/experimental but there are quite a few concepts I'm excited about here. This project came about while writing a program in Go and loving its approach to concurrency. Being a long-time Rubyist I immediately started to think about what similar concepts might look like in Ruby. I set out with two main design constraints: 1. Lightweight: I didn't want routines to be backed by fibers or threads. Having been involved some in the async project (https://github.com/socketry/async), I had some experience using…
- ASA stateful UI runtime for reactive web apps in GoApr 2026 · github.com · ▲14
Doors: Server-driven UI framework + runtime for building stateful, reactive web applications in Go. Some highlights: * Front-end framework capabilities in server-side Go. Reactive state primitives, dynamic routing, composable components. * No public API layer. No endpoint design needed, private temporal transport is handled under the hood. * Unified control flow. No context switch between back-end/front-end. * Integrated web stack. Bundle assets, build scripts, serve private files, automate CSP, and ship in one binary. How it works: Go server is UI runtime: web application runs on a…
- CSCastra – Strip orchestration rights from your LLMsApr 2026 · github.com · ▲8
I got tired of AI agents forgetting what they were doing the moment their context window filled. The current industry solution is to write massively bloated agent harnesses full of defensive spaghetti just to stop models from drifting. The problem is treating chat history as project state. A conversation is not a ledger. Castra is a compiled Go binary that strips orchestration rights from the LLM. State lives in an encrypted, local SQLite database (castra.db). The LLM is just a stateless executor — it reads the DB, executes a highly constrained task, and the result is written back subject to…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
