GPT-4 with Vision Checkup
When new Large Multimodal Models (LMMs) are released, there is excitement as we explore new capabilities. What can a model do? What can't a model do? What strange behaviors does the model exhibit? With that said, such analyses are frozen in time. At a hackathon toward the end of last year, the Roboflow team made a tool that runs the same set of tests with the GPT-4 with Vision API every day. This allows people to see how the model performs over time as updates are made. The last seven days of results are displayed on a web page; the rest of the data is archived in GitHub. We started the site…
GPT-4 with Vision Checkup launched on January 5, 2024 with 9 votes, #1,411 of 4,437 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.
What it does
A collection of experiments measuring the performance of GPT O1 on vision tasks over time.
GPT Checkup has now been moved into Vision AI Checkup, a new tool that benchmarks many SOTA LLMs on vision tasks. This website measures how GPT O1 performs across a range of experiments. We test tasks we know GPT O1 performs well at (i.e. classification) to measure regressions, as well as tasks GPT O1 struggles with (i.e. odometer OCR) to measure performance improvements and changes. You can contribute your own tests, too! See the GitHub README for contributing instructions. Tests are run every day at 1am PT. Last updated April 14, 2025. Today, the average response time to receive results from our tests was 4.28 seconds per request. This number only accounts for requests made by this…from gptcheckup.com
In the maker’s words, at launch
When new Large Multimodal Models (LMMs) are released, there is excitement as we explore new capabilities. What can a model do? What can't a model do? What strange behaviors does the model exhibit? With that said, such analyses are frozen in time. At a hackathon toward the end of last year, the Roboflow team made a…
Does a similar job
all alternatives →
Getting GLM 5.2 running on my slow computerJul 2026 · github.com · ▲937A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
- IRI Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real2023 · sagittarius.greg.technology · ▲434
Hacky Meta Glasses GPT4 Vision Integration2023 · github.com · ▲226v1.0.0: https://github.com/dcrebbin/meta-vision-api/releases/tag/1.0... The previous implementation was very very hacky and used a bookmarklet alongside uglified messenger classes from 2 years ago to monitor a chat, it was also only for image messages. The new release is a browser extension for chrome & firefox that you run on an alt facebook account on messenger that can monitor for new chat & image messages + video monitoring if in a call. You can create a group chat and use a special method (instructions in the readme) to force the Glasses to be aware of…
gpt-realtime-1.5 by OpenAIFeb 2026 · developers.openai.com · ▲300Tighter instruction adherence in speech agents
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, January 2024
the whole month →
Amie▲1,334Turn meeting notes into automated workflows. Handle meetings, summaries, todos and emails (soon) with your AI personal assistant.
Work · 2024 · amie.so
- IM
Sit in Shade - Discover the optimal seat to stay in the shade during your journey. Sit In Shade provides real-time sun exposure data and route visualization.
Life & fun · 2024 · sitinshade.com
