Tarsier – Vision utilities for web interaction agents
Hey HN! I built a tool that gives LLMs the ability to understand the visual structure of a webpage even if they don't accept image input. We've found that unimodal GPT-4 + Tarsier's textual webpage representation consistently beats multimodal GPT-4V/4o + webpage screenshot by 10-20%, probably because multimodal LLMs still aren't as performant as they're hyped to be. Over the course of experimenting with pruned HTML, accessibility trees, and other perception systems for web agents, we've iterated on Tarsier's components to maximize downstream agent/codegen performance. Here's the…
Tarsier – Vision utilities for web interaction agents launched on May 15, 2024 with 192 votes, #271 of 4,539 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Tarsier is a tool that enables language models to understand webpage visual structure through text-based representations rather than images. It converts screenshots into whitespace-structured text using OCR and tags interactive elements with IDs for agents to reference. The tool is designed for developers building web interaction agents and AI codegen systems. According to its creator, text-only models using Tarsier's output outperform multimodal models using screenshots on web automation tasks.
written from the facts on this page · September 2026
From the sources
Vision utilities for web interaction agents 👀. Contribute to reworkd/tarsier development by creating an account on GitHub.
In the maker’s words, at launch
Hey HN! I built a tool that gives LLMs the ability to understand the visual structure of a webpage even if they don't accept image input. We've found that unimodal GPT-4 + Tarsier's textual webpage representation consistently beats multimodal GPT-4V/4o + webpage screenshot by 10-20%, probably because multimodal…
Does a similar job
all alternatives →
GPT-4 vision utilities to browse the web2023 · github.com · ▲10Vision utilities for web interaction agents 👀. Contribute to reworkd/tarsier development by creating an account on GitHub.
Notte Browser Agents2025 · notte.cc · ▲275Hey HN, We’ve been building browser agents for a while. In production, we kept converging on the same pattern: deterministic scripts for the happy path, agents only for edge cases. So we built Demonstrate Mode. The idea is simple: You perform your workflow once in a remote browser. Notte records the interactions and generates deterministic automation code. How it works: - Record clicks, inputs, navigations in a cloud browser - Compile them into deterministic code (no LLM at runtime) - Run and deploy on managed browser infrastructure Closest analog is Playwright codegen but: - Infrastructure…
I wrote an open-source browser alternative for Computer Use for any LLM2024 · github.com · ▲180Hey HN, I made Browser-Use, an open-source tool that lets (all Langchain supported) LLMs execute tasks directly in the browser just with function calling. It allows you to build agents that interact with web elements using natural language prompts. We created a layer that simplifies website interaction for LLMs by extracting xPaths and interactive elements like buttons and input fields (and other fancy things). This enables you to design custom web automation and scraping functions without manual inspection through DevTools. Hasn't this been done a lot of times? Good question, as a general…
- KGKnowledge graph of restaurants and chefs, built using LLMs2025 · theophilecantelob.re · ▲201
Hi HN! My latest side project is knowledge graph that maps the French culinary network using data extracted from restaurant reviews from LeFooding.com. The project uses LLMs to extract structured information from unstructured text. Some technical aspects you may be interested in: - Used structured generation to reliably parse unstructured text into a consistent schema - Tested multiple models (Mistral-7B-v0.3, Llama3.2-3B, gpt4o-mini) for information extraction - Created an interactive visualization using gephi-lite and Retina (WebGL) - Built (with Claude) a simple Flask web app to clean and…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
