nowfound

Life & fun · May 18, 2026

FlashAttention-2 in Cute, from Scratch

A line-by-line dredge-through of Tri Dao

FlashAttention-2 in Cute, from Scratch launched on May 18, 2026 with 6 votes, #1,742 of 20,538 launches that month and more than 75% of that year's launches.

Visit blog.echen.io ↗Alternativestop 8% of May 2026

What it does

Two days. That’s how long it took me to write FlashAttention-2 in Triton. I had never touched a GPU kernel before and somehow replicated a revolutionary algorithm between sips of coffee. I was looking to learn something new and somehow finished before I really got started. I knew the original source was written in CUDA, made by my Bay Area buddy who turned rocks into trillion-ade. Riding on my hubris from my recent success, I decided to translate my pièce de résistance to the native dialect of the rock-whisperers themselves. Three hundred cups of not-Java later, I raised the blood-soaked script in triumph. I spent many moons deciphering the barren scriptures and getting led by astray…from blog.echen.io

Does a similar job

all alternatives →

More life & fun this month

the category →

Source: Show HN post ↗

Launched alongside, May 2026

the whole month →
  • Brew ▲900

    Like Claude design for email marketing

    AI · May 2026 · brew.new

  • Parallel agents, diff reviewer, and multi-model comparisons

    Dev tools · May 2026 · kilo.ai

  • StoreClaw▲806

    Grow your store profits with agents that know how to sell

    AI · May 2026 · storeclaw.ai

  • Give your agent a real number and voice to make calls.

    AI · May 2026 · pollyreach.ai

  • Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…

    Life & fun · May 2026 · github.com

  • 🌱 Private, quiet space for thinking. Simple app for .md files. - zakirullin/files.md

    Dev tools · May 2026 · github.com