Papermusic (draw an instrument, then play it)
This was a fun experiment to try PaliGemma (open vision-language model). I found that PaliGemma performed better than Gemini Flash for this type of specific image task, especially around latency. (~0.9 seconds for PaliGemma inference on a VM, vs. 3-4 seconds for Gemini Flash.) Would love feedback on ways to potentially improve this setup.
In plain words
Papermusic is a web application that converts hand-drawn musical instruments into playable sounds. Users sketch an instrument, and the app uses PaliGemma, an open vision-language model, to recognize the drawing and generate audio. The tool demonstrates PaliGemma's efficiency for image recognition tasks, processing drawings in under one second—significantly faster than comparable alternatives. It's designed for anyone interested in creative music-making or exploring how AI vision models work in practice.
written from the facts on this page · September 2026
Does the same job
all alternatives →
I trained a 125M model to autocomplete piano on-device17d ago · simedw.com · ▲598I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
- IBI built an interactive plotter art exhibit for SIGGRAPH2024 · lostpixels.io · ▲167
I'm enthralled with using pen plotters to make generative art. Last August at SIGGRAPH, I built an interactive experience for others to see how code can be used to make visual art. The linked blog post is my trials and tribulations of linking a MIDI controller to one of these algorithms and sending its output to a plotter, so that people may witness the end-to-end experience.
- AAAn attempt to rethink a music sequencer design2017 · github.com · ▲305
- RARoboPianist, a piano playing robot simulation in the browser2023 · kevinzakka.github.io · ▲260
- FAFree AI-based music demixing in the browser2023 · sevag.xyz · ▲190
Hi all, I've spent some time working on music demixing or music source separation algorithms, which take in a mixed song and output estimates of isolated components (e.g. vocals, drums, bass, other). I took a popular PyTorch model with good performance (Open-Unmix, UMX-L weights), reimplemented the inference steps in C++, and compiled it to WebAssembly for a free client-side music demixer.
More life & fun this month
the category →- TL
Life & fun · 10d ago · louisabraham.github.io
Photosynthesis fires two of your iPhone
Life & fun · 29d ago · photosynthesis.camera
- CCCreatium Coach▲320
Your multimedia mentor that takes you from mid to great
Life & fun · 11d ago · producthunt.creatium.info
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 28d ago · solouno.io
Launched alongside, June 2024
the whole month →
La Growth Machine▲1,216Create personalized, multi-channel conversations at scale
AI · 2024 · lagrowthmachine.com
PyjamaHR▲1,132Hiring on autopilot. The AI applicant tracking system (ATS).
Work · 2024 · pyjamahr.com