Life & fun · alternatives · 2026

24 alternatives to The Hessian of tall-skinny networks is easy to invert

It turns out the inverse of the Hessian of a deep net is easy to apply to a vector. Doing this naively takes cubically many operations in the number of layers (so impractical), but it's possible to do this in time linear in the number of layers (so very practical)! This is possible because the Hessian of a deep net has a matrix polynomial structure that factorizes nicely. The Hessian-inverse-product algorithm that takes advantage of this is similar to running backprop on a dual version of the deep net. It echoes an old idea of Pearlmutter's for computing Hessian-vector products. Maybe this…

This is a computational method for efficiently applying the inverse Hessian to vectors in deep neural networks. Rather than the impractical cubic-time approach, the algorithm exploits the matrix polynomial structure of… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.

1
I invented a new generative model and got accepted to ICLR▲656
I invented Discrete Distribution Networks, a novel generative model with simple principles and unique properties, and the paper has been accepted to ICLR2025! Modeling data distribution is challenging; DDN adopts a simple yet fundamentally different approach compared to mainstream generative models (Diffusion, GAN, VAE, autoregressive model): 1. The model generates multiple outputs simultaneously in a single forward pass, rather than just one output. 2. It uses these multiple outputs to approximate the target distribution of the training data. 3. These outputs together represent a discrete…
2025 · discrete-distribution-networks.github.io · its alternatives →
2
Boldly go where Gradient Descent has never gone before with DiscoGrad▲232
Trying to do gradient descent using automatic differentiation over branchy programs? Or to combine them with neural networks for end-to-end training? Then this might be interesting to you. We develped DiscoGrad, a tool for automatic differentiation through C++ programs involving input-dependent control flow (e.g., "if (f(x) < c) { ... }", differentiating wrt. x) and randomness. Our initial motivation was to enable the use of gradient descent with simulations, which often rely heavily on such discrete branching. The latter makes plain autodiff mostly useless, since it can only account for the…
2024 · github.com · its alternatives →
3
Neural Network Evolution Playground with Backprop NEAT▲73
2016 · blog.otoro.net · its alternatives →
4
Neural Networks and Deep Learning▲95
Core concepts behind neural networks and deep learning
2015 · neuralnetworksanddeeplearning.com · its alternatives →
5
Thinc, a new deep learning library by the makers of spaCy and FastAPI▲250
Thinc is a lightweight type-checked deep learning library for composing models, with support for layers defined in frameworks like PyTorch and TensorFlow.
2020 · thinc.ai · its alternatives →
6OH
Optimizing Higher Order Functions with Hypothetical Inverses▲55
2016 · medium.com · its alternatives →
7
Applying the Unix philosophy to neural networks▲251
Neural network inference the Unix way. Contribute to cloudkj/layer development by creating an account on GitHub.
2019 · github.com · its alternatives →
8
MacMind – A transformer neural network in HyperCard on a 1989 Macintosh▲159
I trained a transformer in HyperCard. 1,216 parameters. 1989 Macintosh. And yes, it took a while. MacMind is a complete transformer neural network, embeddings, positional encoding, self-attention, backpropagation, and gradient descent, implemented entirely in HyperTalk, the scripting language Apple shipped with HyperCard in 1987. Every line of code is readable inside HyperCard's script editor. Option-click any button and read the actual math. The task: learn the bit-reversal permutation, the opening step of the Fast Fourier Transform. The model has no formula to follow. It discovers the…
Apr 2026 · github.com · its alternatives →
9
A radically more scalable alternative to Bayesian Optimization▲96
Experimental Global Optimization Algorithm. Contribute to chrisstroemel/Simple development by creating an account on GitHub.
2018 · github.com · its alternatives →
10
A Implementation of Alpha Zero for Chess in MLX▲71
A chess engine implementation inspired by AlphaZero, using MLX for neural network computations and Monte Carlo Tree Search (MCTS) for move selection.
2025 · github.com · its alternatives →
11
A competition for small neural networks that play strategy gamesfree▲116
A ranked ladder for small neural networks that play strategy games. Train a model, write an adapter, upload two files, and get measured into a weight class from 8 KiB up.
Sep 2026 · tinybrains.dev · its alternatives →
12AG
A GPU-accelerated binary vector index▲65
This is a vector index I built that supports insertion and k-nearest neighbors (k-NN) querying, optimized for GPUs. It operates entirely in CUDA and can process queries on half a billion vectors in under 200 milliseconds. The codebase is structured as a standalone library with an HTTP API for remote access. It’s intended for high-performance search tasks—think similarity search, AI model retrieval, or reinforcement learning replay buffers. The codebase is located at https://github.com/rodlaf/BinaryGPUIndex.
2025 · rlafuente.com · its alternatives →
13
Multi-GPU Reinforcement Learning in Tensorflow for OpenAI Gym▲58
Multi-GPU reinforcement learning using Deep Q-Network in TensorFlow for OpenAI Gym - vishar0/dist-dqn
2016 · github.com · its alternatives →
14
Deep Learning for Program Synthesis▲93
By Rishabh Singh, Jacob Devlin, Abdelrahman Mohamed, and Pushmeet Kohli, Microsoft Research Despite the many advances in computing over the past decades, the actual process of writing computer software has not fundamentally changed — a programmer must manually code the exact algorithmic logic of a p
2017 · microsoft.com · its alternatives →
15
I wrote an autodiff in C++ and implemented LeNet with it▲36
2024 · gitlab.com · its alternatives →
16
Zipslicer, a library for loading LLM checkpoints on consumer hardware▲38
This is a low-level opensource library I developed for my own use and decided to share, as it makes it possible to process large checkpoints of neural networks without renting high-RAM instances, on a regular PC. It replaces torch.load() with a custom function that produces a dictionary that materializes tensors on the fly. Compared to other solutions it doesn't require sharding or re-encoding checkpoints and uses them completely as-is. It is a foundation to make it possible to run inference and compress language models and other large models one layer at a time - in principle, even one…
2023 · github.com · its alternatives →
17BH
Beating Hinton et al.'s capsule net with fewer params and less training▲14
Hello HN, I recently posted a work-in-progress paper, along with code necessary for replicating all its results, at: https://github.com/glassroom/heinsen_routing Among other things, the code in this repo outperforms Hinton et al.'s recent state-of-the-art result in visual recognition[0] while requiring fewer parameters and an order-of-magnitude fewer training epochs. Most of the original research we do at work tends to be either proprietary in nature or tightly coupled to internal code, so we cannot share it with the world. In this case, however, I was able to remove all…
2019 · its alternatives →
18
A weekend replication of STOKE, a stochastic superoptimiser▲58
Haskell re-implementation of STOKE, the stochastic superoptimizer - blaze/notebooks/tutorial.ipynb at master · bollu/blaze
2019 · github.com · its alternatives →
19
Deep learning without gradient descent, 500 layers, no skip connections▲5
Code that reproduces experiments from a scientific paper - xolod7/polyharmonic-cascade
Jan 2026 · github.com · its alternatives →
20
I built a deep learning engine from scratch in Python▲30
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text generation demo trained on the Brown corpus. I'm still an undergrad, so my main goal is to deeply understand how deep learning actually works under the hood - gradients, attention, backpropagation, optimizers - by building it step-by-step with full visibility into everything, and without…
2025 · github.com · its alternatives →
21
Train a Convolutional Neural Nets Using Multi-Party Computation▲28
Most efforts to privately train machine-learning models either use homomorphic encryption or secure multi-party computation (MPC). Recent insights indicate…
2018 · blog.openmined.org · its alternatives →
223W
3d Word Vectors with Neural Networks, T-SNE, and WebGL▲30
2013 · wordcloud.ersatz1.com · its alternatives →
23IC
I created a Neural Network from scratch, in scratch▲7
I spent the past week implementing a 1 Layer Neural Net and training it on MNIST within the visual scripting language provided by scratch.mit.edu. It was tedious, but ultimately not too difficult. The code runs incredibly slowly, so much so that 64 samples of MNIST takes 5+ hours to train on my machine. There were a lot of little mini challenges that were fun to overcome (implementing softmax was very tricky). If you're interested, I encourage you to try and improve on it! More details in the linked blog post.
2024 · bell-boy.github.io · its alternatives →
24
Attention-Based Guided Structured Sparsity of Deep Neural Networks▲5
Attention-Based Guided Structured Sparsity of Deep Neural Networks - astorfi/attention-guided-sparsity
2018 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →