Flash Attention in ~100 lines of CUDA
Flash Attention in ~100 lines of CUDA (forward pass only) - tspeterkim/flash-attention-minimal
Flash Attention in ~100 lines of CUDA launched on March 16, 2024 with 230 votes, #203 of 4,270 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Flash Attention is a CUDA implementation that optimizes attention mechanisms in machine learning models through a streamlined, minimal codebase of roughly 100 lines. It targets developers and researchers working with neural networks who need faster attention computation on GPUs. What stands out is its simplicity and efficiency—achieving significant performance improvements while remaining compact and understandable, making it accessible for those wanting to understand or modify attention optimization at a low level.
written from the facts on this page · September 2026
Does a similar job
all alternatives →
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning2023 · github.com · ▲385Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
Mini-AGI – Dynamic continual learning model trained on 8GB VRAMSep 2026 · github.com · ▲277Continual learning model trained from scratch on 8GB VRAM laptop with batch-1 stream of data. - volotat/mini-AGI
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
- IMI made a minimalist spaced repetition tool2020 · memordo.com · ▲140
More life & fun this month
the category →
A free city-building game in your browser. You are the mayor. You know what the city needs. Now get it past the zoning board, the neighbors, the appeals court and the next election.
Life & fun · 3d ago · permit.petercai.com
- IV
Life & fun · 15d ago · tanzaniaeducationproject.org
Turn a phone, tablet or TV into a sign. Type a message and it fills the screen: welcome signs, timers, Wi-Fi passwords and more. No app, no account, free.
Life & fun · 16d ago · bigwords.page
Play a free virtual guitar online with your keyboard or by touch. Slow songs down and see every chord, string and strum. Acoustic, classic, electric and bass.
Life & fun · 10d ago · guitoro.com
Carrier settings from iPhone, Pixel and Galaxy firmware, decoded and compared: APNs, VoLTE, 5G and Wi-Fi Calling per carrier, and what each build changed.
Life & fun · 14d ago · carrierexplode.com
Life & fun · 24d ago · space.bl2.net
Source: Show HN post ↗
Launched alongside, March 2024
the whole month →
🌐 The Internet Computer! Free, Open-Source, and Self-Hostable. - HeyPuter/puter
Dev tools · 2024 · github.com
Microlaunch▲1,115Launch and get feedback on both the idea and product
Dev tools · 2024 · microlaunch.net
