AnyModal – Train Your Own Multimodal LLMs
I’ve been working on AnyModal, a framework for integrating different data types (like images and audio) with LLMs. Existing tools felt too limited or task-specific, so I wanted something more flexible. AnyModal makes it easy to combine modalities with minimal setup—whether it’s LaTeX OCR, image captioning, or chest X-ray interpretation. You can plug in models like ViT for image inputs, project them into a token space for your LLM, and handle tasks like visual question answering or audio captioning. It’s still a work in progress, so feedback or contributions would be great. GitHub:…
AnyModal – Train Your Own Multimodal LLMs launched on November 17, 2024 with 8 votes and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
AnyModal is a Flexible Multimodal Language Model Framework for PyTorch - ritabratamaiti/AnyModal
AnyModal is a modular framework for building multimodal language models in PyTorch. It bridges pretrained vision (or other modality) encoders with large language models via pluggable projector architectures — letting you train a multimodal model with just a config file and a few lines offrom github.com
In the maker’s words, at launch
I’ve been working on AnyModal, a framework for integrating different data types (like images and audio) with LLMs. Existing tools felt too limited or task-specific, so I wanted something more flexible. AnyModal makes it easy to combine modalities with minimal setup—whether it’s LaTeX OCR, image captioning, or chest…
Does a similar job
all alternatives →
LLM-aided OCR – Correcting Tesseract OCR errors with LLMs2024 · github.com · ▲479Almost exactly 1 year ago, I submitted something to HN about using Llama2 (which had just come out) to improve the output of Tesseract OCR by correcting obvious OCR errors [0]. That was exciting at the time because OpenAI's API calls were still quite expensive for GPT4, and the cost of running it on a book-length PDF would just be prohibitive. In contrast, you could run Llama2 locally on a machine with just a CPU, and it would be extremely slow, but "free" if you had a spare machine lying around. Well, it's amazing how things have changed since then. Not only have models gotten a lot better,…
ApertureDB Multimodal AI Workflows2025 · aperturedata.io · ▲144Automate common AI tasks for multimodal data
I just open sourced my document/website extractor for Vision-LLMs2024 · github.com · ▲37Hi HackerNews, Lately, I have seen an explosion in posts offering paid APIs/services to get unstructured data into LLMs (i.e. langchain extract, ragflow, unstructured, unstract, just to name a few) and I have been largely disappointed by them, either because they fail to implement multimodal support, fail to give good context for "really tricky" PDFs / Word docs / Powerpoints, or are just plain difficult to use. In light of all these posts I figured I'd share my solution that has been working smoothly for me and my clients. I put it up on GitHub for free so you can check it…
Multimodal perception system for real-time conversationFeb 2026 · raven.tavuslabs.org · ▲54I work on real-time voice/video AI at Tavus and for the past few years, I’ve mostly focused on how machines respond in a conversation. One thing that’s always bothered me is that almost all conversational systems still reduce everything to transcripts, and throw away a ton of signals that need to be used downstream. Some existing emotion understanding models try to analyze and classify into small sets of arbitrary boxes, but they either aren’t fast / rich enough to do this with conviction in real-time. So I built a multimodal perception system which gives us a way to encode visual…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, November 2024
the whole month →
Lovable▲1,477Hey HN! Previous CERN physicist turned hacker here. We've developed a way to make AI coding actually work by systematically identifying and fixing places where LLMs typically fail in full-stack development. Today we're launching as Lovable (previously gptengineer.app) since it's such a big change. The problem? AI writing code typically make small mistakes and then get stuck. Those who tried know the frustration. We fixed most of this by mapping out where LLMs fail in full-stack dev and engineering around those pitfalls with prompt chains. Thanks to this, in all comparisons I found with: v0,…
AI · 2024 · lovable.dev
Postiz▲1,188Streamline your social media with Postiz. Schedule posts, analyze performance, and manage all accounts in one place.
Dev tools · 2024 · postiz.com
I wasn't quite sure if this qualified as "Show HN" given you can't really download it and try it out. However, dang said[0]: > If it's hardware or something that's not so easy to try out over the internet, find a different way to show how it actually works—a video, for example, or a detailed post with photos. Hopefully I did that? Additionally, I've put code and a detailed guide for the netboot computer management setup on GitHub: https://github.com/kentonv/lanparty Anyway, if this shouldn't have been Show HN, I apologize! [0]…
Dev tools · 2024 · lanparty.house
Raycast Notes▲1,058Unlock a new level productivity with AI, Cloud Sync and more
Work · 2024 · raycast.com
