Revibing nanochat's inference model in C++ with ggml
Recently I wanted to see if I could vibe some serious C++ code. The result is a C++ re-implementation of Andrej Karpathy's nanochat's inferencing part (https://github.com/karpathy/nanochat), built on top of ggml. Unlike llama.cpp, this isn't a standalone binary; it is a C++ library & Python wrapper designed to swap out some core classes within the nanochat pipeline. For playability, I’ve tried to keep the dependencies to a minimum: just ggml, nanobind, and gtest for unit tests. Features and limitations: - A drop-in replacement of nanochat’s `GPT` and `KVCache` classes. So…
Revibing nanochat's inference model in C++ with ggml launched on January 10, 2026 with 5 votes, #3,400 of 14,584 launches that month and more than 75% of that year's launches.
What it does
nanochat's inference engine re-vibed in C++ with GGML. - k-ye/nanochagg.ml
A C++ inference engine for nanochat models, built on top of GGML . This project is a personal hobby—a mix of fun coding, learning, and experimenting with modern AI agents. Do NOT use it in production (even if you are actually deploying nanochat in production...). git clone [email protected]:k-ye/nanochagg.ml.git cd nanochagg.ml # Don't forget this! git submodule update --init --recursive You need to have at least one nanochat model checkpoint first. I've used nanochat-d32 extensively during development. uv pip install -e /path/to/nanochagg.ml Developer Installation For contributors working on nanochaggml: # Create env and install dependencies (CPU PyTorch by default) uv sync --dev # Or for GPU…from github.com
In the maker’s words, at launch
Recently I wanted to see if I could vibe some serious C++ code. The result is a C++ re-implementation of Andrej Karpathy's nanochat's inferencing part (https://github.com/karpathy/nanochat), built on top of ggml. Unlike llama.cpp, this isn't a standalone binary; it is a C++ library & Python…
Does a similar job
all alternatives →
GPT-4.1 in the API2025 · openai.com · ▲375Announcing GPT-4.1, GPT-4.1 mini, & GPT-4.1 nano in the API
GPT‑5.4 mini and nanoMar 2026 · openai.com · ▲267Fast and efficient models optimized for coding and subagents
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
Port of OpenAI's Whisper model in C/C++2022 · github.com · ▲399Hi HN, OpenAI recently released a model for automatic speech recognition called Whisper [0]. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++. The entire code is less than 8000 lines of code and is contained in just 2 source files without any third-party dependencies. The Github project is here: https://github.com/ggerganov/whisper.cpp With this implementation I can very easily build and run the model - “make…
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
