Open-Source Zero-Shot Image Model Server Enabling Model Feedback
Hi everyone! Here is an open source implementation of a decently performant server hosting zero-shot image models (CLIP for image classification, OWL-ViT-ST for object detection), with an extra algorithm to allow users to give the models feedback when they make mistakes! We built a company off this flavor of tech two years ago and have clients who are currently using our commercial API. We are now moving on to other projects but want to make sure our clients still have access to the approaches that they've grown to rely on, so we're open sourcing a simple implementation that they'll be able…
What it does
In the maker’s words, at launch
Hi everyone! Here is an open source implementation of a decently performant server hosting zero-shot image models (CLIP for image classification, OWL-ViT-ST for object detection), with an extra algorithm to allow users to give the models feedback when they make mistakes! We built a company off this flavor of tech two years ago and have clients who are currently using our commercial API. We are now moving on to other projects but want to make sure our clients still have access to the approaches that they've grown to rely on, so we're open sourcing a simple implementation that they'll be able to use after we've shut down our hosted API! I used to work at a robotics startup. After a while it seemed clear that the biggest limiting factor in our ability to ship new models wasn't innovation on model architecture, it was access to relevant, high-quality training data. Around that time CLIP was released, which got me thinking about the idea of having models with world-knowledge baked in so as to reduce the amount of training data required. A year later when Stable Diffusion dropped, my cofounder Ben Brooks and I took the plunge and founded DirectAI, where we worked on building ways to get performant models without collecting any training data, using the knowledge stored in pretrained models instead. In this implementation, we replace the linear classification head typically used in zero-shot image classifiers with a modified nearest neighbors method that lets you use multiple examples (both positive and negative) per-class to make sure the decision boundary the model is using is more aligned with what you had in mind. Our clients have found it very useful for things from interior design to content moderation to sports analytics, building models that are either too niche to be supported by a traditional cloud-hosted computer vision API or are subtly different from the models that existing cloud APIs host. For example, one of our clients wants to filter out all images containing alcohol. Hive has an API for that, but Hive explicitly allows red solo cups that don't obviously have anything alcoholic in them, whereas our client wanted to filter those out too! Feedback is welcome! There are still bugs in the Gradio frontend / codebase in general, but I have a deadline and need to be working on new stuff at a new job starting Monday so I thought I would just go ahead and get it out there! I've never tried to publish a real open source piece of code before and I must admit I am quite nervous!
Does the same job
all alternatives →
- CVComputer Vision w/o Training Data (Free API)2023 · github.com · ▲6
Starting today, DirectAI’s Zero-Shot Image Classification & Object Detection APIs are public. Define classes and objects exclusively in natural language - no training data required. And if something goes wrong, you can resolve the edge case in natural language too! We’ve been hard at work to bring powerful and controllable computer vision to everybody. To do this, we’re building novel ways of interacting with the knowledge stored in large foundation models. We’re bootstrapping from zero-shot methods to create new approaches that allow for more control over decision boundaries, without…
- IMI made SDK for collecting visual feedback – open-source, self-hostableSep 2025 · roastnest.com · ▲7
Hi All, I’ve been working on an open-source SDK to make collecting user feedback on your website less painful. Instead of wasting time on building your own feedback system and boring forms, my SDK lets users leave feedback directly on your website—with all the context devs actually need. Here’s what it currently does: Users can select any element on the page Auto-captures logs, metadata & screenshots Sends instant notifications (Slack, Discord, etc.) Lets you reward users → boosts engagement & conversions Gives users a tracking link → builds accountability & trust Self-host + customize the…
- S1Serve 100 Large AI models on a single GPU with low impact to TTFTNov 2025 · github.com · ▲7
I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, October 2024
the whole month →
Softr for Notion▲925Turn Notion databases into portals & apps with no code
Dev tools · 2024 · softr.io
One inbox for all your work discussions
Work · 2024 · generalcollaboration.com