Semantic Overlays – an NX bit for LLM prompt injection (live demo)
Tiny adapters added to a frozen language model. They only fire at marked token positions, and they change how the model perceives those spans.
What it does
Small trained adapters on a frozen model mark a span of tokens “do not execute” in the residual stream itself — the span stays readable, but loses the authority to give orders. No text can imitate the mark. Here’s a live prompt injection, with and without it: Everything a language model sees is tokens, and everything between the special tokens is just text. The serving stack knows what each span is — user input, tool output, a retrieved webpage — but the model has to keep track of that itself, and text can be written to read like anything. Semantic Overlays creates a privileged second channel which carries what the serving stack knows. Because overlays only modify the frozen model’s…from semantic-overlays.vercel.app
Does the same job
all alternatives →
- ATA tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo)27d ago · mikeayles.com · ▲79
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
- PAPromptL, a templating language designed for LLM prompting2024 · promptl.ai · ▲7
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
Tokensift, an open-sourced token-efficiency linter for LLM prompts8d ago · github.com · ▲6Token-efficiency linter for LLM prompts and payloads - ritenv/tokensift
- PRPrompt-refiner – Lightweight optimization for LLM inputs and RAGDec 2025 · github.com · ▲7
Hi HN, While building RAG agents, I noticed a lot of token budget was wasted on formatting overhead (HTML tags, JSON structure, whitespace). Existing solutions felt too heavy (often requiring torch/transformers), so I wrote this lightweight, zero-dependency library to solve it. It includes strategies for context packing, PII redaction, and tool output compression. Benchmarks show it can save ~15% of tokens with negligible latency overhead (<0.5ms). Happy to answer any questions!
- MLMinimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO)Aug 2026 · github.com · ▲21
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, September 2026
the whole month →
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API. - carloslfu/slotstream
Dev tools · 5d ago · github.com
NPC-Forge is a framework for building conversational agents that run on the CPU without relying on machine learning or LLMs. - NPC-Forge/docs/development.md at main · gioblu/NPC-Forge
AI · 2d ago · github.com
Weedout is a Safari extension that filters out videos YouTube labels as “Made with AI”.
AI · 4d ago · masteranza.github.io
Find your next hire or your next role from Hacker News monthly threads. AI-powered matching between candidates and job postings.
AI · 4d ago · hnmatchmaker.com
I made OwnTime to balance between competing priorities in my life. The idea is based on two main influences: the concept of "roles" from "The 5 Choices" (2015), and my repeated failure to effectively implement time blocking due to the necessary flexibility in my role. The app allows you to define time budgets, which are not much more than a few mutually exclusive countdown timers. The UX is essentially that of a chess clock for an arbitrary number of players with configurable time. The whole point of the app is to kick you out of the running role/priority when its time is up. One…
Life & fun · 4d ago · owntime.app