Nevermind XOR – Deep Learning Has an Issue with Sin
More precisely, even the best neural networks can not be trained to approximate periodic functions using stochastic gradient descent. (empirically, prove me wrong!) Contents Simulating a clock Simplifying the problem Why can’t we train the sin? What to do about it? Simulating a clock The problem tha
Nevermind XOR – Deep Learning Has an Issue with Sin launched on August 21, 2022 with 34 votes, #829 of 3,307 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 4 years on.
In plain words
Nevermind XOR is a deep learning educational project that explores how neural networks struggle with certain mathematical functions, particularly sine waves. Launched in 2022, it examines the limitations of standard deep learning approaches when applied to trigonometric problems. The project is designed for students and researchers interested in understanding the practical challenges and quirks of training neural networks. It serves as a learning resource for those exploring the intersection of machine learning and mathematical functions.
written from the facts on this page · September 2026
From the sources
More precisely, even the best neural networks can not be trained to approximate periodic functions using stochastic gradient descent. (empirically, prove me wrong!) Contents Simulating a clock Simplifying the problem Why can’t we train the sin? What to do about it? Simulating a clock The problem that led me into this sinkhole is an attempt to model biological rhythms. People sleep every 24 hours,
Does a similar job
all alternatives →
How I topped the HuggingFace open LLM leaderboard on two gaming GPUsMar 2026 · dnhkng.github.io · ▲495I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1. As of 2026, the top 4 models on that leaderboard are still descendants. The weird finding: single-layer duplication does nothing. Too few layers, nothing. Too many, it gets worse. Only circuit-sized blocks of ~7 layers work. This suggests pretraining carves out discrete functional circuits in the layer stack that only work when preserved whole. The whole thing was developed on 2x RTX 4090s in my basement. I'm…
Netmind Power2023 · netmind.ai · ▲272Hi HN, We just opened public access to Qwen3‑Coder‑480B‑A35B‑Instruct via a REST/SDK API. Why it might be interesting: - Fully open-source (Apache‑2.0): https://github.com/QwenLM/Qwen3-Coder - 480B params (35B active MoE), long context up to 1M tokens with YaRN - Matches Claude Sonnet/GPT‑4o on SWE-Bench Verified (81.6%) - Free tier: 100K tokens/month
A perceptron with stochastic gradient descent in Go2017 · github.com · ▲94A single / multi layer / recurrent neural network written in Golang. - made2591/go-perceptron-go
- IBI built a toy TPU that can do inference and training on the XOR problem2025 · tinytpu.com · ▲134
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…
MacMind – A transformer neural network in HyperCard on a 1989 MacintoshApr 2026 · github.com · ▲159I trained a transformer in HyperCard. 1,216 parameters. 1989 Macintosh. And yes, it took a while. MacMind is a complete transformer neural network, embeddings, positional encoding, self-attention, backpropagation, and gradient descent, implemented entirely in HyperTalk, the scripting language Apple shipped with HyperCard in 1987. Every line of code is readable inside HyperCard's script editor. Option-click any button and read the actual math. The task: learn the bit-reversal permutation, the opening step of the Fast Fourier Transform. The model has no formula to follow. It discovers the…
Deep Learning for Program Synthesis2017 · microsoft.com · ▲93By Rishabh Singh, Jacob Devlin, Abdelrahman Mohamed, and Pushmeet Kohli, Microsoft Research Despite the many advances in computing over the past decades, the actual process of writing computer software has not fundamentally changed — a programmer must manually code the exact algorithmic logic of a p
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗
Launched alongside, August 2022
the whole month →
DhiWise 2.0▲740Convert Figma designs into Flutter 3.0 & React code rapidly
Dev tools · 2022 · dhiwise.com
- PA
Hey HN, I've been working on https://pornpen.ai, a site for generating adult images. Please only visit the site if you are 18+ and willing to look at NSFW images. This site is an experiment using newer text-to-image models. I explicitly removed the ability to specify custom text to avoid harmful imagery from being generated. New tags will be added once the prompt-engineering algorithm is fine-tuned further. If the servers are overloaded, take a look at the feed and search pages to look through past results. For comments/suggestions/feedback please visit…
AI · 2022 · pornpen.ai