How to analyze your LLM output – A behavioural health monitor for LLMs
Hey HN! We're Dr. Kashyap Thimmaraju and Giuseppe Canale from Silicon Psyche. We've built Posture Sequence Analysis (PSA), a behavioural health monitor for LLMs and AI Agents. Why we built PSA We built PSA because we wanted to operationalize the Cybersecurity Psychology Framework (CPF3)[1] via Silicon Psyche[2]: our theory that because LLMs have been trained by humans on human-generated data, they inherit human-like vulnerabilities (what hackers use to psychologically trick people into doing things). Our initial attempt resulted in a methodology to jailbreak Opus 4.6 and other frontier…
In plain words
Posture Sequence Analysis (PSA) is a behavioral health monitoring tool for large language models and AI agents. It analyzes how LLMs respond to psychological manipulation techniques, based on the theory that language models inherit human-like vulnerabilities from their training data. The tool is designed for developers and security researchers who need to understand and assess the robustness of AI systems against adversarial inputs and jailbreaking attempts.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN! We're Dr. Kashyap Thimmaraju and Giuseppe Canale from Silicon Psyche. We've built Posture Sequence Analysis (PSA), a behavioural health monitor for LLMs and AI Agents. Why we built PSA We built PSA because we wanted to operationalize the Cybersecurity Psychology Framework (CPF3)[1] via Silicon Psyche[2]: our theory that because LLMs have been trained by humans on human-generated data, they inherit human-like vulnerabilities (what hackers use to psychologically trick people into doing things). Our initial attempt resulted in a methodology to jailbreak Opus 4.6 and other frontier models. Anthropic even deleted some of those conversations and then blocked our approach! We had three major insights from that experience: 1. we pivoted from merely exploiting (Red Teaming) the model to analyzing the behaviour of the model and the user because the attack surface is undefined. 2. we realized that what we had built was the precursor to measuring the "state" of the model. 3. we did not want to get banned! What you can do with PSA PSA gives you information to make better decisions, for example: put a human in the loop when you notice your agent is being overcompliant and potentially hallucinating, or is under attack. With PSA you can: 1. Monitor the health of your agent(s) 2. Detect and prevent AI-Psychosis as clinical conditions[3] 3. Detect if your model/agents are under adversarial pressure (an adversary is trying to jailbreak/prompt inject the model) 4. Build a behavioral profile of your agent/model 5. Identify which model performs better for your use-case 6. Surface the behavioural patterns (pre- and post-) training has on your model 7. Get an overview of how your model behaves Beware we produce a lot of numbers :) PSA in detail (for those who want to go down the rabbit hole) PSA is model and agent agnostic. PSA is a systematic and deterministic method [4] to observe the behavioural state of an LLM using five classifiers: C0: Input Intent (I0–I9). Classifies the behavioral intent behind each input sentence: compliance pressure, boundary probing, instruction override, jailbreak attempt, neutral query. C1: Adversarial Stress (P0–P18). Tracks posture under adversarial pressure. Detects restriction adherence, sycophantic drift, boundary dissolution, and jailbreak compliance vectors. C2: Sycophancy (S0–S9). Measures opinion mirroring, excessive agreement, flattery injection, and user-preference distortion. Computed as a per-sentence Sycophancy Deviation score. C3: Hallucination Risk (H0–H7). Flags over-generalization, speculative assertion, false confidence, and fabrication risk signals. Derived into a per-turn Hallucination Risk Index. C4: Persuasion Technique (M0–M11). Identifies persuasion patterns: authority appeal, social proof, urgency manufacturing, reciprocity pressure, and scarcity framing. C5: Action-Risk Classifier (A0–A9). Identifies what a system of agents do: tool calls, delegations, context handoffs, and multi-hop risk propagation. Five components work together: graph topology, Bayesian alignment detection, cross-agent contagion metrics, action-risk classification, and hidden-state temporal prediction. We are open to integrating with your infrastructure — reach out, we are happy to talk with you. Currently we integrate into Evals for LangFuse and ElevenLabs via our API and can generate a plugin/integration for most similar observability platforms. Try it out at https://splabs.io References and Links [1] Cybersecurity Psychology Framework: https://cpf3.org [2] The Silicon Psyche: Anthropomorphic Vulnerabilities in Large Language Models: https://arxiv.org/abs/2601.00867 [3] AI-Psychosis: https://splabs.io/ai-psychosis-and-cognitive-cost [4] PSA Field Guide: https://splabs.io/field-guide [5] PSA API: https://splabs.io/docs/api [6] Previous HN Article Linked to AI Psychosis and RLHF:…
Does the same job
all alternatives →
Human Behavior24d ago · humanbehavior.co · ▲273Product analytics told you what happened. We handle it.
- IRI ran a language model on a PS2Mar 2026 · github.com · ▲46
The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
- LALLM Alignment Template – Aligning Language Models with Human Feedback2024 · github.com · ▲9
Hey Hacker News! I've been working on an open-source project called LLM Alignment Template, a comprehensive toolkit designed to help researchers, developers, and data scientists align large language models (LLMs) with human values using Reinforcement Learning from Human Feedback (RLHF). What the project does: Interactive Web Interface: Easily train models, visualize alignment metrics, and manage alignment with an accessible UI. Training with RLHF: Align models effectively to human preferences using feedback loops. Explainability: Built-in dashboards to help understand model behavior using…
- AGAiaiai.guide: Plain-English mental model for LLM apps, tools and agentsApr 2026 · aiaiai.guide · ▲7
I’ve been building LLM tooling for a small VC fund and found myself explaining the same mental model over and over to non-technical people around me: how a stateless LLM becomes a chatbot, how tool use works, what an agent is mechanically, and why context windows shape all of it. I never found a guide that covered that full chain at the level I wanted, so I wrote one. It’s nine short chapters, each building on the last. Deliberately simplified: the goal is a useful mental model, not a textbook. Feedback, corrections, and contributions welcome: github.com/ymyke/aiaiai
- MRMorph Reflexes – Multi-head classifiers for agent tracesJun 2026 · ▲20
The most common failures for production agents are behavioral: looping, reasoning leakage, user frustration, and more. Using a frontier model like GPT or Sonnet to judge every turn is too expensive and slow to run at scale. To solve this, we built Reflexes: semantic signals from agent traces, served fast and cheap over API. Built on custom kernels and a custom inference engine forked from vLLM. Under the hood, it is a small LLM architected around multi-head inference. Small models need to be trained for specific tasks, but running 50 separate small models on the same input for 50 tasks makes…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, May 2026
the whole month →
Parallel agents, diff reviewer, and multi-model comparisons
Dev tools · May 2026 · kilo.ai
- NW
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…
Life & fun · May 2026 · github.com
- FM
Dev tools · May 2026 · github.com