Duplicate 3 layers in a 24B LLM, logical deduction .22→.76. No training
I replicated David Ng's RYS method (https://dnhkng.github.io/posts/rys/) on consumer AMD GPUs (RX 7900 XT + RX 6950 XT) and found something I didn't expect. Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act as indivisible cognitive units. Duplicate the right block and the model runs its reasoning pipeline twice. No weights change. No training. The model just thinks longer. The results on standard benchmarks (lm-evaluation-harness, n=50): Devstral-24B, layers 12-14 duplicated once: - BBH Logical Deduction: 0.22 → 0.76…
Duplicate 3 layers in a 24B LLM, logical deduction .22→.76. No training launched on March 18, 2026 with 265 votes, #108 of 23,380 launches that month and more than 99% of that year's launches.
In plain words
This project demonstrates a technique for improving large language model reasoning by duplicating specific layer blocks without retraining. By duplicating contiguous groups of 3-4 layers in models like Devstral-24B, the approach enables models to run their reasoning processes multiple times using unchanged weights. Testing shows significant improvements on logical reasoning and mathematical tasks, with logical deduction accuracy jumping from 0.22 to 0.76 and code generation improving from 0.72 to 0.78. The method works on consumer AMD GPUs and requires no model fine-tuning.
written from the facts on this page · September 2026
From the sources
I replicated Ng's RYS method and found that duplicating 3 specific layers in Qwen2.5-32B boosts reasoning by 17% and duplicating layers 12-14 in Devstral-24B improves logical deduction from 0.22→0.76 on BBH — no training, no weight changes, just routing hidden states through the same circuit twice. Tools included. Two AMD GPUs, one evening. - alainnothere/llm-circuit-finder
In the maker’s words, at launch
I replicated David Ng's RYS method (https://dnhkng.github.io/posts/rys/) on consumer AMD GPUs (RX 7900 XT + RX 6950 XT) and found something I didn't expect. Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act as indivisible cognitive units.…
Does a similar job
all alternatives →
How I topped the HuggingFace open LLM leaderboard on two gaming GPUsMar 2026 · dnhkng.github.io · ▲495I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1. As of 2026, the top 4 models on that leaderboard are still descendants. The weird finding: single-layer duplication does nothing. Too few layers, nothing. Too many, it gets worse. Only circuit-sized blocks of ~7 layers work. This suggests pretraining carves out discrete functional circuits in the layer stack that only work when preserved whole. The whole thing was developed on 2x RTX 4090s in my basement. I'm…
I built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
- ABAutoThink – Boosts local LLM performance with adaptive reasoning2025 · ▲397
I built AutoThink, a technique that makes local LLMs reason more efficiently by adaptively allocating computational resources based on query complexity. The core idea: instead of giving every query the same "thinking time," classify queries as HIGH or LOW complexity and allocate thinking tokens accordingly. Complex reasoning gets 70-90% of tokens, simple queries get 20-40%. I also implemented steering vectors derived from Pivotal Token Search (originally from Microsoft's Phi-4 paper) that guide the model's reasoning patterns during generation. These vectors encourage behaviors like numerical…
Netmind Power2023 · netmind.ai · ▲272Hi HN, We just opened public access to Qwen3‑Coder‑480B‑A35B‑Instruct via a REST/SDK API. Why it might be interesting: - Fully open-source (Apache‑2.0): https://github.com/QwenLM/Qwen3-Coder - 480B params (35B active MoE), long context up to 1M tokens with YaRN - Matches Claude Sonnet/GPT‑4o on SWE-Bench Verified (81.6%) - Free tier: 100K tokens/month
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
Littlebird▲709Littlebird remembers everything across your meetings, messages, docs, and apps, then takes action so you can recall any detail and never lose context.
AI · Mar 2026 · littlebird.ai
