nowfound

Life & fun · December 30, 2023

Moondream, a small vision language model that runs on 8GB of RAM

I've been working on training this small vision language model for the last month - excited to release the first prototype today! It is based on SigLIP (image encoder), Phi-1.5 (text model) and trained using the LLaVa-1.5 training dataset. It runs reasonably fast on CPU with ~8GB of RAM in full 32-bit precision. There's plenty of room to speed it up and reduce memory consumption by quantizing the model. I posted a video of it running on my M2 Macbook Air (on CPU not MPS, so performance should be comparable on other hardware) on Twitter to demonstrate inference speed:…

Moondream, a small vision language model that runs on 8GB of RAM launched on December 30, 2023 with 9 votes, #1,087 of 3,780 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.

Visit github.com ↗Alternatives

What it does

tiny vision language model. Contribute to m87-labs/moondream development by creating an account on GitHub.

In the maker’s words, at launch

I've been working on training this small vision language model for the last month - excited to release the first prototype today! It is based on SigLIP (image encoder), Phi-1.5 (text model) and trained using the LLaVa-1.5 training dataset. It runs reasonably fast on CPU with ~8GB of RAM in full 32-bit precision.…

Does a similar job

all alternatives →
  • I built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915

    Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.

  • Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419

    Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…

  • Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205

    Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm

  • Moonshine Open-Weights STT models – higher accuracy than WhisperLargev3Feb 2026 · github.com · ▲316

    I wanted to share our new speech to text model, and the library to use them effectively. We're a small startup (six people, sub-$100k monthly GPU budget) so I'm proud of the work the team has done to create streaming STT models with lower word-error rates than OpenAI's largest Whisper model. Admittedly Large v3 is a couple of years old, but we're near the top the HF OpenASR leaderboard, even up against Nvidia's Parakeet family. Anyway, I'd love to get feedback on the models and software, and hear about what people might build with it.

  • DeepSeek-V4Apr 2026 · huggingface.co · ▲387

    The open-source era of 1M context intelligence

  • IB
    I built a toy TPU that can do inference and training on the XOR problem2025 · tinytpu.com · ▲134

    We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…

More life & fun this month

the category →

Source: Show HN post ↗ · launch text quoted in part

Launched alongside, December 2023

the whole month →
  • Beeper Mini – iMessage client for Android▲1,521

    Hi HN! I’m proud to share that we have built a real 3rd party iMessage client for Android. We did it by reverse engineering the iMessage protocol and encryption system. It's available to download today (no waitlist): https://play.google.com/store/apps/details?id=com.beeper.ima and there's a technical writeup here: https://blog.beeper.com/p/how-beeper-mini-works. Unlike every other attempt to build an iMessage app for Android (including our first gen app), Beeper Mini does not use a Mac server relay in the cloud. The app connects directly to Apple…

    Work · 2023 · beeper.com

  • Heynote – A dedicated scratchpad for developers▲1,063

    Hey! I made Heynote entirely for my own use case. For many years, I always had an Emacs instance running with the scratch buffer open, even long after I had abandoned Emacs as my programming editor in favor of more recent IDE:s. The simplicity of having just one big scratch buffer appeals to me, but I still want to separate the different things I jot down somehow (without using tabs or similar). Previously, my solution was to insert a bunch of blank lines between the notes, but hitting C-A would still select the entire buffer. That's why I came up with the concept of "blocks", which turned…

    Work · 2023 · heynote.com

  • LogoFast▲988

    Make beautiful logos with AI, fast & free

    AI · 2023 · logofa.st

  • FuseBase▲939

    Client collaboration platform for professional services

    Dev tools · 2023 · thefusebase.com

  • Beep! 2.0▲871

    Fastest tool to assign and share visual tasks with your team

    Work · 2023 · justbeepit.com

  • Inbox Zero▲821

    Clean Up Your Inbox In Minutes Newsletter management, AI automation, and email analytics. Inbox Zero is the open-source email app that puts you back in control of your inbox.

    AI · 2023 · getinboxzero.com