OmniParser V2

In plain words

OmniParser V2 converts UI screenshots into structured, machine-readable elements that large language models can understand and interact with. By parsing visual interfaces into interpretable components, it enables LLMs to identify and execute actions on screen elements without native computer control capabilities. The tool transforms any LLM into a functional computer use agent capable of navigating applications and interfaces based on visual input.

written from the facts on this page · September 2026

From the sources

OmniParser ‘tokenizes’ UI screenshots from pixel spaces into structured elements in the screenshot that are interpretable by LLMs. This enables the LLMs to do retrieval based next action prediction given a set of parsed interactable elements.

Yadong Lu, Senior Researcher; Thomas Dhome-Casanova (opens in new tab), Software Engineer; Jianwei Yang, Principal Researcher; Ahmed Awadallah, Partner Research Manager Graphic User interface (GUI) automation requires agents with the ability to understand and interact with user screens. However, using general purpose LLM models to serve as GUI agents faces several challenges: 1) reliably identifyi

Graphic User interface (GUI) automation requires agents with the ability to understand and interact with user screens. However, using general purpose LLM models to serve as GUI agents faces several challenges: 1) reliably identifying interactable icons within the user interface, and 2) understanding the semantics of various elements in a screenshot and accurately associating the intended action with the corresponding region on the screen. OmniParser closes this gap by ‘tokenizing’ UI screenshots from pixel spaces into structured elements in the screenshot that are interpretable by LLMs. This enables the LLMs to do retrieval based next action prediction given a set of parsed interactable…from microsoft.com

• 
  I wrote an open-source browser alternative for Computer Use for any LLM2024 · github.com · ▲180

  Hey HN, I made Browser-Use, an open-source tool that lets (all Langchain supported) LLMs execute tasks directly in the browser just with function calling. It allows you to build agents that interact with web elements using natural language prompts. We created a layer that simplifies website interaction for LLMs by extracting xPaths and interactive elements like buttons and input fields (and other fancy things). This enables you to design custom web automation and scraping functions without manual inspection through DevTools. Hasn't this been done a lot of times? Good question, as a general…

• 
  AnyParser API2024 · cambioml.com · ▲885

  The first LLM for document parsing with accuracy and speed

• 
  OmniParser2024 · ▲4

  OmniParser, UI automation, AI screen analysis, UI testing

• 
  OmniworkAug 2026 · omniwork.ai · ▲418

  The Creative Agent OS, create better with desktop AI agents

• 
  AI Observability by OpenObserveAug 2026 · openobserve.ai · ▲317

  OpenTelemetry-native observability for agents and LLMs

• 
  Turn any website into a knowledge base for LLMs2024 · embedding.io · ▲305

  I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 17d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 16d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 18d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 20d ago · stillwet.art

• 
  Onepin▲384

  AI voiceover with every name, price and date said right

  AI · 30d ago · onepin.ai

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 28d ago · world.pixverse.video

• 
  Screen Studio 3.0▲1,832

  Beautiful screen recordings with instant shareable links

  Growth · 2025 · screen.studio

• 
  Tana▲1,665

  Put your notes to work with voice and AI

  AI · 2025 · tana.inc

• IG
  I got laid off from Meta and created a minor hit on Steam▲1,581

  I was at FB/Meta from late 2013 to early 2023, mostly working in the compiler/runtime spaces. I got hit in the spring 2023 layoff wave. I immediately started making games in my newfound free time (a lifelong interest, and I even worked in AA(A?) back ca. ~2000), and in October 2023 I stumbled upon the idea of a roguelike pachinko/plinko game inspired by Luck Be A Landlord. Things snowballed quickly, I started talking to publishers, then worked like crazy through all of 2024, almost the hardest I've ever worked in my career, and launched the game in December 2024. It's sold…

  Work · 2025

• 
  Tanka▲1,410

  AI Messenger with smart reply & long term memory for teams

  AI · 2025 · tanka.ai

• 
  Tool Finder▲1,327

  Your shortcut to better software

  AI · 2025 · toolfinder.com

• 
  I built an app to stop me doomscrolling by touching grass▲1,203

  i wanted to change the habit of reaching for my phone in the morning and doomscrolling away an hour so i built an app to help me. now i have to literally touch grass before accessing my most distracting apps the app is built in swiftui, uses the screen time apis provided by apple and google vision to recognise grass or not i'd love to get your thoughts on the concept.

  Life & fun · 2025 · touchgrass.now