Alternatives
Products that do what Linear RNN/Reservoir hybrid generative model, one C file (no deps.) does
I just noticed it takes literally ~5 minutes to train millions parameters on slow CPU...but before you call Yudkowsky that "it's over", an important note: the main bottleneck is the corpus size, params are just 'cleverness' but given limited info it's powerless. Anyway, here is the project: https://github.com/bggb7781-collab/lrnnsmdds/tree/main couple of notes: 1. single C file, no dependencies. Below are literally all the "dependencies", not even custom header (copy paste from the top of the single c file): #define _POSIX_C_SOURCE 200809L #include #include…
- 1IB
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
Apr 2026 · github.com
- 2II
I invented Discrete Distribution Networks, a novel generative model with simple principles and unique properties, and the paper has been accepted to ICLR2025! Modeling data distribution is challenging; DDN adopts a simple yet fundamentally different approach compared to mainstream generative models (Diffusion, GAN, VAE, autoregressive model): 1. The model generates multiple outputs simultaneously in a single forward pass, rather than just one output. 2. It uses these multiple outputs to approximate the target distribution of the training data. 3. These outputs together represent a discrete…
Oct 2025 · discrete-distribution-networks.github.io
- 3SU
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org
- 4SF
Hey folks! We're Alex and Evan, and we're working on putting together a 512 H100 compute cluster for startups and researchers to train large generative models on. - it runs at the lowest possible margins (<$2.00/hr per H100) - designed for bursty training runs, so you can take say 128 H100s for a week - you don’t need to commit to multiple years of compute or pay for a year upfront Big labs like OpenAI and Deepmind have big clusters that support this kind of bursty allocation for their researchers, but startups so far have had to get very small clusters on very long term contracts, wait…
2023 · sfcompute.org
- 5IB
Hi HN, Over the past few months, I've been building `dsc`, a tensor library from scratch in C++/CUDA. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally. The key features are: - C++ core with CUDA support written from scratch. - A familiar, PyTorch-like Python API. - Runs real models: it's complete enough to load a model like Qwen from HuggingFace and run inference on both CUDA and CPU with a single line change[1]. - Simple, built-in observability for both Python and C++. Next on the roadmap is…
2025 · github.com
- 6N5
2020 · nn-512.com
- 7WM
Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…
2024 · glhf.chat
- 8AC
built a tiny pytorch clone in c after going through prof. vijay janapa reddi's mlsys book: mlsysbook.ai/tinytorch/ perfect for learning how ml frameworks work under the hood :)
Dec 2025 · github.com
- 9FR
Morning HN. Random number generation feels is a somewhat underrepresented topic in the C++ realm. There is a lot of questionable info about it found online and even the standard library is quite behind the times in terms of it's algorithms. It suffers from trying to accommodate sometimes impractical standard requirements and has several ways of getting significantly bad statistical results. This leaves a lot easily achievable performance & quality on the table. So, being a mathematician who mostly works with stochastic models and wants these models to run fast and well, I embarked on a…
2025 · github.com
- 10FG
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://arxiv.org/abs/2311.14479. Feedback and potential applications are welcome.
2023 · github.com
- 11MM
Hi HN! We (Thomas and Stéphan, hello!) recently released Model2Vec, a Python library for distilling any sentence transformer into a small set of static embeddings. This makes inference with such a model up to 500x faster, and reduces model size by a factor of 15 (7.5M params or 15/30MB on disk, depending on whether you use float16 or float32). This allows you to embed 50-100k documents per second on a cpu on a macbook. This reduction of course comes at a cost: distilled models are worse than their parent models. Even so, they are actually a lot better than large sets of conventional…
2024 · github.com
- 12ML
We’ve recently open-sourced Model2vec, a method to distill sentence transformers into static embeddings that outperform all previous approaches by a large margin on MTEB. Our new models set a new state-of-the-art for static embeddings. Main features: - Our best model (potion-base-8M) has only 8M parameters, which is ~30mb on disk - Inference is ~500x faster than the distilled base model (bge-base), on a CPU - New models can be distilled in 30 seconds on a CPU without requiring a dataset - just a vocabulary - Numpy-only inference: The packaged can be install the package with minimal…
2024 · github.com
- 13BA
Hello HN! I want to share something me and a few friends have been working on for a while now — Zeroshot, a web tool that builds image classifiers using text-image models and autolabeling. What does this mean in practice? You can put together an image classifier in about 30 seconds that’s faster and more accurate than CLIP, but that you can deploy yourself however you’d like. It’s open source, commercially licensed, and doesn’t require you to pay anyone per API call. Here's a 2 minute video that shows it off: https://www.youtube.com/watch?v=S4R1gtmM-Lo How/why does it…
2023 · usezeroshot.com
- 14
As the title and description of the GitHub repo suggest, I’m working on a small project for purely educational purposes, with the goal of implementing generative model inference (small models capable of modeling 2D distributions) based on the Flow Matching paradigm in C. I’ve worked on generative AI models based on Flow Matching from a more “abstract” perspective, using frameworks like PyTorch, and I wanted to understand what goes on behind the scenes. The repository is still a work in progress and is also one of my first "serious" projects in C.
Jul 2026 · github.com
- 15
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com
- 16MR
Hey HN! We’ve just open-sourced model2vec-rs, a Rust crate for loading and running Model2Vec static embedding models with zero Python dependency. This allows you to embed text at (very) high throughput; for example, in a Rust-based microservice or CLI tool. This can be used for semantic search, retrieval, RAG, or any other text embedding usecase. Main Features: - Rust-native inference: Load any Model2Vec model from Hugging Face or your local path with StaticModel::from_pretrained(...). - Tiny footprint: The crate itself is only ~1.7 mb, with embedding models between 7 and 30 mb. Performance:…
2025 · github.com
- 17CM
I'm a machine learning engineer who always found it annoying to integrate ML models into phone apps, smartwatch apps, microcontroller firmware etc... Why do we need all these libraries and runtimes with all the overhead, compatibility issues and other headaches, when it's just some math to be executed? So I made a compiler that simply converts the model into plain source code with no dependencies, and it actually solved all my deployment problems. Now I'm curious if it can help anyone else too. Through the link you can submit your model file (Keras h5, onnx soon to be supported), and I'll…
2023 · waveworks.dk
- 18
I wanted to know how fast a 26B mixture-of-experts model could run on a desktop CPU with no GPU. Got ~40 tok/s single-stream (lossless) and ~124 batched. The surprising part was the byte budget: for this model you compress the output head (32% of per-token bytes), not the experts (16%). The writeup has the bandwidth roofline and the dead-ends; the repo has the reproducible recipe. Happy to answer questions. Repo: https://github.com/arun-prasath2005/gemma4-cpu-moe
Jun 2026 · apeg.dev
- 19MM
Hi HN! We (Thomas and Stéphan, hello!) recently released Model2Vec, a Python library for distilling any sentence transformer into a small set of static embeddings. This makes inference with such a model up to 500x faster, and reduces model size by a factor of 15 (7.5M params or 15/30MB on disk, depending on whether you use float16 or float32). This reduction of course comes at a cost: distilled models are a lot worse than their parent models. Even so, they are actually a lot better than large sets of conventional static embeddings, such as GLoVe or word2vec-based models, which are many…
2024 · github.com
- 20OS
Hey HN! A few months ago we shared our AI dataset generator as an open source repo, and the response was incredible (https://news.ycombinator.com/item?id=44388093). We got requests from folks who wanted to use it without the hosting overhead, so we created both options: a hosted version (https://www.metabase.com/ai-data-generator for instant use and the source code fully open (https://github.com/metabase/dataset-generator) for anyone who wants to self-host or contribute. Looking forward to seeing how you use it and what you build on top of…
Sep 2025 · metabase.com
- 21IC
I spent the past week implementing a 1 Layer Neural Net and training it on MNIST within the visual scripting language provided by scratch.mit.edu. It was tedious, but ultimately not too difficult. The code runs incredibly slowly, so much so that 64 samples of MNIST takes 5+ hours to train on my machine. There were a lot of little mini challenges that were fun to overcome (implementing softmax was very tricky). If you're interested, I encourage you to try and improve on it! More details in the linked blog post.
2024 · bell-boy.github.io
- 22IW
2024 · gitlab.com
- 23UA
Hello! So, I am experimenting with new approaches to automated machine learning, where you don't need anything besides either data or prompt This is a first platform from my auto ml serie called Łukasiewicz (Jan Łukasiewicz was a logician who created Polish notation) I much appreciate your feedback and if you want to try it out, I created a -75% off promo code "HACKERNEWS" on credits, so it hopefully will cover at least some of computing expenses, but more importantly will let you play with the platform at almost no cost Thank you!
2024 · lukasiewicz.tmlc.pl
- 24SG
Hi HN, I built NOMA (Neural-Oriented Machine Architecture), a systems language where reverse-mode autodiff is a compiler pass (lowered to LLVM IR). My goal is to treat model parameters as explicit, growable memory buffers. Since NOMA compiles to standalone native binaries (no Python runtime), it allows using realloc on weights mid-training. This makes "self-growing" architectures a system primitive rather than a complex framework hack. I just pushed a reproducible benchmark (Self-Growing XOR) to validate the methodology: it compares NOMA against PyTorch and C++, specifically testing how…
Dec 2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →