AI audio embeddings - Discover 100M+ iTunes songs with language models
We’re a group of electronic music artists and PhD researchers working in AI music. We've been working on a project to help us discover new music in a more objective way. It's called Speak Music: https://speakmusic.sonophase.com/ We’ve trained an AI model to understand the correspondence between music and language. The model combines a machine listening and audio signal processing with transformers for text embeddings. Once trained, we index a huge catalogue of unseen audio, ensuring that the search system can efficiently scale to millions of tracks. At the moment, our model is…
What it does
In the maker’s words, at launch
We’re a group of electronic music artists and PhD researchers working in AI music. We've been working on a project to help us discover new music in a more objective way. It's called Speak Music: https://speakmusic.sonophase.com/ We’ve trained an AI model to understand the correspondence between music and language. The model combines a machine listening and audio signal processing with transformers for text embeddings. Once trained, we index a huge catalogue of unseen audio, ensuring that the search system can efficiently scale to millions of tracks. At the moment, our model is optimised for our preferred music; electronic, techno, ambient, dub and relaxing etc. We’re currently fine-tuning to handle all kinds of genres and moods. Our model enables two types of discovery: (a) natural language search and (b) similarity search. (a) Speak: search for music using freeform natural language prompts. Describe the mood, aesthetic, texture, setting and context of a track. (b) Music: discover tracks that are acoustically similar to ANY reference track from Apple Music. We recently presented at Sonar+D in Barcelona. We hope you like it. Let us know what you think!
Does the same job
all alternatives →- ITI trained an AI model on 120M+ songs from iTunes2023 · maroofy.com · ▲753
Hey HN! I just shipped a project I’ve been working on called Maroofy: https://maroofy.com You can search for any song, and it’ll use the song’s audio to find other similar-sounding music. Demo: https://twitter.com/subby_tech/status/1621293770779287554 How does it work? I’ve indexed ~120M+ songs from the iTunes catalog with a custom AI audio model that I built for understanding music. My model analyzes raw music audio as input and produces embedding vectors as output. I then store the embedding vectors for all songs into a vector database, and use semantic…
- SASonauto – A more controllable AI music creator2024 · sonauto.ai · ▲454
Hey HN, My cofounder and I trained an AI music generation model and after a month of testing we're launching 1.0 today. Ours is interesting because it's a latent diffusion model instead of a language model, which makes it more controllable: https://sonauto.ai/ Others do music generation by training a Vector Quantized Variational Autoencoder like Descript Audio Codec (https://github.com/descriptinc/descript-audio-codec) to turn music into tokens, then training an LLM on those tokens. Instead, we ripped the tokenization part off and replaced it with a normal…
- MAMusic Audio Search Engine Using OpenAI's Embeddings on GPT Descriptions2023 · muzic-sage.vercel.app · ▲92
I am excited to announce a new tool for music producers and audio enthusiasts - a music audio search engine. With just a simple description of the groove you're looking for, our semantic search engine will output the most similar audio in seconds. I used the Freesound.org API to upload over 3,000 grooves to MongoDB, and combined all the relevant data such as tags, title, description, BPM, etc. into OpenAI's Text-Davinci to generate a unique description of each sound. I then embedded these descriptions using the Ada Embeddings Model and inserted them into Pinecone DB vector database, making…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, June 2023
the whole month →- NI
Heya HN, I've been working on a reddit-like platform as my primary side project for the last few years. Doing a (very) soft launch today, mainly because I want to use it to encourage discussion of alternatives. How non.io works: 1. Free to browse, paid to interact. 2. Minimum subscription is $2 (though you can choose more). I take $1 to run the servers, everything left gets split evenly between everything you upvote that month. It's a simple model, but I hope it's a better one than the freemium model we've been relying on for the last few years. Fundamentally I feel like any ad-supported…
Commerce · 2023 · non.io
Typefully 2.0▲931Effortlessly publish on Twitter and LinkedIn, now with AI
Growth · 2023 · typefully.com
SlashPage▲760Create websites in seconds to build your first 1K community
Growth · 2023 · slashpage.com
