Alternatives

Products that do what Example POC to run the Dolly V2 12B model in 8bit mode does

I banged this out today because nobody seems to have figured this out yet (and I wanted to play with this model). I was inspired by the fact that Simon Willison was struggling to get this running[0], so I wanted to take a crack at it! This is by no means "clean", but it is a baseline for others to hack from. Feel free to send over PRs and I'll land them! (I'd love it somebody used this to glue this into this webui[1].) 0: https://til.simonwillison.net/llms/dolly-2 1: https://github.com/oobabooga/text-generation-webui/

1IU
I used AI to recreate a $4000 piece of audio hardware as a plugin▲160
Hi Hacker News, This is definitely out of my comfort zone. I just wanted to show you guys because I'm super proud of it. It's a 100% faithful recreation based off of the schematics, patents, and ROMs that were found online. So please watch the video and tell me what you think https://youtu.be/auOlZXI1VxA The reason why I think this is relevant is because I've been a programmer for 25 years and AI scares the shit out of me. I'm not a programmer anymore. I'm something else now. I don't know what it is but it's multi-disciplinary, and it doesn't involve writing code myself--for…
Jan 2026
2IR
I ran a language model on a PS2▲46
The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
Mar 2026 · github.com
3PU
Plock: Use a local LLM from anywhere in your OS▲6
After seeing a cool demo of a hack on Twitter, I built a cross platform version of it that works well and uses streaming. From anywhere on Mac and Linux, trigger Ollama and optionally feed it your clipboard. I built it yesterday and it's already very useful to me. I'm pretty excited about it and wanted to share!
2024 · github.com
4FT
Fun things to do with your VM/370 machine▲20
Hi All. I made this as an fun intro to help people who have zero IBM mainframe experience and no access to a modern IBM mainframe (at least, not access to do whatever you want with it). I appreciate tips, suggestions and anything that might improve the experience for someone who has no idea of how those machines operate(d).
Jan 2026 · rbanffy.github.io
5IB
I built a website where you can easily fine-tune Llama 3.1 models▲8
Hey HN, I built a website where you can train Llama 3.1 8b & 70b (4bit) on your data. I use unsloth in the backend and the training is done on H100s which I rent programmatically from Runpod. I'd love some feedback. If you would be interested in using it feel free to book a chat with me: cal.com/hamada/tunellama-intro Happy to give you free credits :) P.S. I'm also looking for a co-founder as I have big plans for this.
2024 · tunellama.com
6RA
Run any Llama model finetune and more, instantly▲7
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
7IB
I built an Open Source screen timer for the m5stickc (Arduino)▲17
I've never posted on ShowHN before, but I wanted to share my Xmas 2025 project; to try a new approach to controlling our kids screen time. This also involved massively over-engineering a solution in order to play with a shiny new gadget (and avoid the in laws at Christmas, obviously) I've shared some learnings on AI coding with embedded devices, and how I approached the product design/hardware selection side of things. The Web App is at https://screenie.org - and I'm Open Sourcing the device and web app code later today (links to follow on that site)
Jan 2026 · partridge.works
8AF
A framebuffer console viewer, pure Go▲7
Another Sunday afternoon project: add some pizzaz to your boring bare metal or virtual machine consoles! Written as a single Go binary; no dependencies. https://github.com/Fusion/modernfbv As usual, feedback please!
2022
9IM
I made a VS Code extension where you can use Mixtral 8x7B for free▲8
Hey guys, I made a VS Code extension, within which you can use the Mixtral 8x7B AI model for free. Mixtral 8x7B is a new AI model from the Mistral family, which outperforms GPT-3 in many benchmarks, a fact that I have personally confirmed in daily coding tasks, where Mistral 8x7B was the clear winner. You can use all the AI features, such as AI Lens with custom instructions and similar. I'm pretty sure some of features will be useful to you. I'd appreciate both, positive and negative feedback!
2023 · marketplace.visualstudio.com
10R5
Run 500B+ Parameter LLMs Locally on a Mac Mini▲17
Hi HN, I built OpenGraviton, an open-source AI inference engine that pushes the limits of running extremely large LLMs on consumer hardware. By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini. Early benchmarks: TinyLlama-1.1B drops from ~2GB (FP16) to ~0.24GB with ternary quantization. At 140B scale, models that normally require ~280GB fit within ~35GB packed. Optimized for Apple Silicon with Metal + C++ tensor unpacking, plus…
Mar 2026 · github.com
11
Shoehorn, a library to quantize an LLM to fit your Mac's VRAM▲6
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
23d ago · github.com
12YA
Yet another tool to wrap agents in a loop▲6
I was randomly browsing claude codes ralph wiggum plugin[1] and was surprised to see my toy project referenced. Since it appears people are getting some value out of it, thought I'd share it here... Full disclosure, this repo itself was built with a primitive ralph wiggum loop so expect AI slop. It works though. ¯\_(ツ)_/¯ https://mikeyobrien.github.io/ralph-orchestrator/ [1] https://github.com/anthropics/claude-code/tree/main/plugins/...
Dec 2025 · github.com
13MO
My open-source liquid handler ( biohacking )▲18
Excuse my typing, I've just been up for a few hours and I've made my deadline for SB 5.0 =] ... but for HN.. you guys get it early I present to you my fellow hackers: https://github.com/delinquentme/LH001 ....I call it the 'LH001' , it is: open source, arduino based, 96 well microplate, peristaltic pump, liquid handler running: open protocol controls ( in browser or over wire ), In Ruby on Rails And here it is in person: http://www.youtube.com/watch?v=ZY5IY5CZ1es Questions? Critiques? Thoughts? So, do I have HN's permission to change my twitter bio to include "biohacker" ?
2011
14IB
I built a tool to check if your computer can run LLMs locally▲8
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
15RV
RISCY-V02: A 16-bit 2-cycle RISC-V-ish CPU in the 6502 footprint▲7
Finally finished my little CPU project, RISCY-V02. I built it (with Claude) to challenge the notion that the 6502 was a "local optimum" in its transistor budget. Given the constraints of 1970s home computers (~1 MHz DRAM, so raw clock speed doesn't help), could RISC have been a better design choice? This design argues yes: pipelining, barrel shifters, and more registers beat microcode PLAs, questionable addressing modes, and hardware BCD. Highlights: 8x 16-bit general-purpose registers (vs 3x 8-bit on 6502) 2-stage pipeline (Fetch/Execute) with speculative fetch 61 fixed 16-bit…
Mar 2026 · github.com
16IJ
I just open-sourced my web based POS software – WallacePOS▲10
Hi Hacker News, Early last year I decided to build a web based POS solution. A charity I was working with at the time tasked me with finding a affordable POS solution. The system had to support multiple locations at a reasonable price. Unfortunately I was unable to find any software that fit and so after an initial proof of concept, the project was born. I aimed at using the latest technologies used in the browser and had fun working with them (localstorage, websockets, etc), as well as some very tricky but gratifying problem solving. About a year and a half later I feel WallacePOS is ready…
2015
17LA
L88 – A Local RAG System on 8GB VRAM (Need Architecture Feedback)▲12
Hey everyone, I’ve been working on a project called L88 — a local RAG system that I initially focused on UI/UX for, so the retrieval and model architecture still need proper refinement. Repo: https://github.com/Hundred-Trillion/L88-Full I’m running this on 8GB VRAM and a strong CPU (128GB RAM). Embeddings and preprocessing run on CPU, and the main model runs on GPU. One limitation I ran into is that my evaluator and generator LLM ended up being the same model due to compute constraints, which defeats the purpose of evaluation. I’d really appreciate feedback on:…
Feb 2026
18YA
Yet another "my first open-source project" thread. Feedback, please?▲7
I must say I was encouraged to post my project here when I saw the positive response to this post: http://news.ycombinator.com/item?id=3780731 Before you make any comments, please keep in mind that the project is written for fun. I'm not forcing anyone to use it. In fact, nobody should use the current unstable version in a serious project. Here is the link: http://github.com/bsg/libserv Please share your thoughts. All criticisms are welcome, even the harsh ones. p.s. The project is still a work-in-progress.
2012
19GA
Gochan – A library of channel architectures for Go, inspired by Rust▲5
Hi All, I felt like I was re-inventing the wheel by bolting similar channel architectures onto different Go structs repeatedly so I decided to extract some common types into into one library so that they would be easier to re-use: - oneshot - spsc - spmc - mpsc - mpmc - broadcast - watch The types are inspired by Rust channels so if you're coming from Rust they should feel familiar. So far I'm really enjoying using them but it'd be great to get some external feedback if you have time! https://github.com/amorey/gochan Andres
May 2026 · github.com
20IM
I'm building a simple way to package, collaborate, and deploy ML models▲9
Heya HN, after spending +1 year building an ML-driven analytics product (that didn't pan out unfortunately), I've pivoted to solving a problem my team and I found while building the previous product … why the hell is it so hard to move a model from a Jupyter notebook, to a development server, then to a production pipeline!? To solve this my team and I started the open source KitOps project under the Apache 2 license. KitOps includes the Kit CLI that uses a Kitfile manifest to create ModelKits: 1. The kit CLI packages your model, datasets, code, and configuration into an OCI compliant…
2024 · kitops.ml
21OR
OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini▲13
Hi HN, I built OpenGraviton, an open-source AI inference engine designed to push the limits of running extremely large models on consumer hardware. The system combines several techniques to drastically reduce memory and compute requirements: • 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic sparsity with Top-K pruning and MoE routing • mmap-based layer streaming to load weights directly from NVMe SSDs • speculative decoding to improve generation throughput These allow models far larger than system RAM to run locally. In early benchmarks, OpenGraviton reduced…
Mar 2026 · opengraviton.github.io
22IA
I Am Making an 8-Bit CPU in Logisim▲5
Hey HN, I am making an 8-Bit CPU in Logisim and blogging about it.t's nothing fancy, really simple, unoptimized dumb CPU for sake of learning. I am no electrical engineer, I am just a self-taught dude who has programming as a hobby and wanted to dive deeper into abstractions. So here it is, so far I wrote 2 parts: [links redacted] I welcome any and all critique and feedback, as I can be wrong about everything for sure.
2017
23IS
I successfully failed at one-shot-ing a video codec like h.264▲9
Read an article yesterday about the H.264 codec increasing their licensing fee by an astronomical amount. And as always, my first shot was how hard could it be to try and build a codec which could be that efficient. I've personally been on a drive to improve my ability to one-shot complex features, products, or make even surgical changes. It's been a few months since I've been doing that, and honestly, results have been great for both work and work/life balance. This was a fun experiment. It burned through tokens, but it helped me identify some more improvements I could make to my…
Apr 2026 · github.com
24IV
I Vibe Coded Doge Decimator in Websim [video]▲6
I recently made this game Doge Decimator with the websim team, and wanted to make a process video for the steps I took to make it in an AI-native platform! You can check out every single iteration the site went through from the links in the description :)
2025 · youtube.com

Ranked by how close each launch is in meaning, then by votes. Refine with a description →