Hlb-CIFAR10 0.2.0: New world record (~<12.38s) on single-GPU CIFAR10
Hello everyone, After recreating the accuracy/rough speed from David Page's implementation in hlb-CIFAR10 0.1.0 (18.1s on an A100, SXM4, Colab), it was down to some basic NVIDIA kernel profiling to figure out which operations were the long poles in the tent. Perhaps (somewhat?) unsurprisingly, the NCHW NHWC thrash was the worst part, but unfortunately the GhostBatchNorm was a barrier even using the faster-on-Ampere channels_last memory format. A quick note before continuing -- some may find the use of a convolutional network and on CIFAR10 to be curious. A quick answer to that would be…
What it does
In the maker’s words, at launch
Hello everyone, After recreating the accuracy/rough speed from David Page's implementation in hlb-CIFAR10 0.1.0 (18.1s on an A100, SXM4, Colab), it was down to some basic NVIDIA kernel profiling to figure out which operations were the long poles in the tent. Perhaps (somewhat?) unsurprisingly, the NCHW <-> NHWC thrash was the worst part, but unfortunately the GhostBatchNorm was a barrier even using the faster-on-Ampere channels_last memory format. A quick note before continuing -- some may find the use of a convolutional network and on CIFAR10 to be curious. A quick answer to that would be that in doing the research that optimizes well-known problems (especially if the testing path is incredibly rapid), we get much clearer pictures of what certain fundamental information learning limits are for systems like this, as well as stable prototypes that can then be translated (potentially somewhat analogously) into other modalities. You can see this practice with a few researchers, Hinton comes to mind though his work is much more fundamental and experimental than this is. Back to the release notes. Ultimately, however, we were able to get a similar level of regularization to the original GhostBatchNorm (called GhostNorm) in the code, which allowed us to remove it and a bunch of tensor allocation/contiguous tensor calls, saving us nearly exactly 5 seconds or so (!!!!). Replacing the call for nn.AdaptiveMaxPooling(1,1) with a torch.amax(dim=2,3), added an additional .5 seconds off the clock, bringing us down below Thomas Germer (@99991)'s excellently quick implementation of the same base method (https://github.com/99991/cifar10-fast-simple) and giving us the new world record. This work is pretty simple on its own -- though the various ways to use the nvidia profiler(s) can be very daunting to use and I can post snippets of the simplest way that I've found (via the torch.profiler route) if someone asks/is curious. That said, looking at kernel execution order and times can really and truly do a lot to quickly improve a network in conjunction with good research engineering practices. This is what I'm pretty good at doing so getting to flex a bit on a spare time project is fun. I'm consistently storing up time saves into a draft bin of sorts and plan on keeping releasing them in related/clustered releases as I'm able to appropriately polish them to whatever their capabilities seem to be. There is a lot of room to grow, and I think we now definitely have a good chance at making it within that 94% accuracy under ~<2s mark within a few years! This work is meant to be a living resume for me, feel free to check out my README.md for more info. I love a lot of aspects of the technical/nitty gritty side of the fusion of neural network engineering and the edge of research, particularly when it comes to speed, so this is my strong area. I'm certainly happy to answer whatever reasonable questions anyone might have, let me help with getting this project going for you (or other related stuff -- feel free to ask! <3 :)))) )
Does the same job
all alternatives →
- 1T15FPS to 60FPS, new GPU real-time flow-based method2020 · youtube.com · ▲112
- AMA Minimal Code Example of Using Vulkan for Computations on the GPU2017 · github.com · ▲68
- HIHellepoll is a blazingly-fast async HTTP server written in C++2011 · github.com · ▲35
This is based on an approach I made for a very high performance message and video multiplexing server. The project died, but I'm glad I get to share the a basic version with some http test code now. I admit this is the second time I've tried to show this off here today; I posted earlier but it got lost when no-one was watching :(
More life & fun this month
the category →- TL
Life & fun · 10d ago · louisabraham.github.io
Photosynthesis fires two of your iPhone
Life & fun · 28d ago · photosynthesis.camera
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 28d ago · solouno.io
Scroll through all 43,252,003,274,489,856,000 reachable Rubik's Cube permutations.
Life & fun · 26d ago · everycube.alen.is
Hi HN, I built Eigendrum, a web tool that solves the 2D wave equation for arbitrary shapes so you can hear what they sound like as drums. How it works: * Solves -∇²u = λu using finite element analysis (Kφ = λMφ) on a triangle mesh. * Validated to <0.1% error against closed-form solutions for circles (Bessel zeros) and rectangles. * Sound model factors in strike location, Rayleigh damping, and mallet width. * Includes Kac drums I & II to demonstrate identical sound spectra from different geometries. * No frameworks, build steps, or dependencies. Repo and tests:…
Life & fun · 27d ago · baselashraf81.github.io
Launched alongside, January 2023
the whole month →- NA
This works suprisingly well. Just give it instructions like "make it winter" or "remove the cars" and the photo is altered. Here are some examples of transformations it can make: Golden gate bridge: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... Girl with a pearl earring: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... I integrated this new InstructPix2Pix model into imaginAIry (python library) so it's easy to use for python developers.
AI · 2023 · github.com
HyperSwitch▲715Fast, reliable, and affordable open source payments switch
Dev tools · 2023 · hyperswitch.io
- IV
The project is called QueryStorm. It uses Roslyn to offer C# (and VB.NET) support in Excel, as an alternative to VBA. I've posted about it before, but a lot has changed since then so figured I'd share an update. The current version includes a host of new features, namely a C# debugger, support for NuGet packages, and the ability to publish Excel extensions to an "AppStore" (which is essentially a NuGet repository). The AppStore can be used by anyone with the (free) runtime component. Another great addition is the community license, which is a free license for individuals and small companies…
Work · 2023 · querystorm.com
- IM
Hey HN! Some more info: I’m an NHS doctor and the founder of Pi-A (https://www.pi-a.io) which developed Lungy (https://www.lungy.app). Lungy is an app (iOS only for now) that responds to breathing in real-time and was designed to make breathing exercises more engaging and beneficial to do. It hopefully has many aspects of interest to the HN community – real-time fluid, cloth and soft body sims running on the phone’s GPU. My background is as a junior surgical trainee and I started building Lungy in 2020 during the first COVID lockdown in London. During COVID, there were…
Life & fun · 2023 · lungy.app
