Skyvern 2.0 – open-source AI Browser Agent scoring 85.8% on WebVoyager
Hey HN, We’re Suchintan and Shu from Skyvern (https://www.skyvern.com). We’re building an open source AI Agent that can browse the web and take actions. Our open source repo can be found at https://github.com/Skyvern-AI/Skyvern. We’ve re-built Skyvern with a Planner-Actor-Validator agent architecture and achieved 85.8% state of the art (SOTA) on the WebVoyager Benchmark. You can see the results for yourself here: https://eval.skyvern.com/ For reference, here were the previous SOTA results: 83.5% - Google Mariner…
What it does
In the maker’s words, at launch
Hey HN, We’re Suchintan and Shu from Skyvern (https://www.skyvern.com). We’re building an open source AI Agent that can browse the web and take actions. Our open source repo can be found at https://github.com/Skyvern-AI/Skyvern. We’ve re-built Skyvern with a Planner-Actor-Validator agent architecture and achieved 85.8% state of the art (SOTA) on the WebVoyager Benchmark. You can see the results for yourself here: https://eval.skyvern.com/ For reference, here were the previous SOTA results: 83.5% - Google Mariner (https://deepmind.google/technologies/project-mariner/) 73.1% - AgentE (https://arxiv.org/html/2407.13032v1) 67.0% - HCompany (https://www.hcompany.ai/blog/a-research-update) 59.1% - WebVoyager (https://arxiv.org/html/2401.13919v4) 52.6% - WILBUR (https://arxiv.org/html/2404.05902v1) 52.0% - Claude Computer Use (https://docs.anthropic.com/en/docs/build-with-claude/compute...) Achieving this SOTA result required expanding Skyvern’s original architecture. Skyvern 1.0 involved a single prompt operating in a loop both making decisions and taking actions on a website. This approach was a good starting point, but scored ~45% on the WebVoyager benchmark because it had insufficient memory of previous actions and could not do complex reasoning. We re-built this all using a Planner-Actor-Validator agent architecture: 1. Planner - Decides that goals to accomplish on a website, and maintains a working memory of the overall goal and progress towards it 2. Actor - Given a narrowly scoped goal, executes the goal on the website, reporting back 3. Validator - Asserts whether the goal was successfully achieved and passes feedback back to the Actor + Planner We ran the benchmark on Skyvern cloud to test Skyvern 2.0 in a real-world environment – autonomously navigating the web in a remotely hosted browser without any human involvement. To keep with our open source mission, we decided to publish benchmark, modifications, and final results for anyone to review. This is important because we’re seeing an increasing trend of companies publishing their benchmarks with no way to access the results, so we’ve decided to make everything public. [1] Eval Dataset: https://github.com/Skyvern-AI/skyvern/tree/main/evaluation/d... [2] Modifications: https://github.com/Skyvern-AI/skyvern/pull/1576/commits/60dc... [3] Each run (incl prompts + responses) can be inspected here: https://eval.skyvern.com/ The full report (incl an architecture diagram) can be found here: https://blog.skyvern.com/skyvern-2-0-state-of-the-art-web-na... If you’d like to give Skyvern a try, you can grab the open source version (https://github.com/Skyvern-AI/Skyvern) or the cloud version (https://app.skyvern.com/) and give it a go and share any feedback with us. We look forward to any and all of your comments!
Does the same job
all alternatives →
- SBSkyvern – Browser automation using LLMs and computer vision2024 · github.com · ▲422
Hey HN, we're building Skyvern (https://www.skyvern.com), an open-source tool that uses LLMs and computer vision to help companies automate browser-based workflows. You can see some examples here: https://github.com/Skyvern-AI/skyvern#real-world-examples-of... and there's a demo video at https://github.com/Skyvern-AI/skyvern#demo, along with some instructions on running it locally. We provide a natural-language API to automate repetitive manual workflows that happen within the companies' backoffices. You can check out our code and play with…
- INIndex – New Open Source browser agent2025 · github.com · ▲98
Hey HN, Robert from Laminar (lmnr.ai) here. We built Index - new SOTA Open Source browser agent. It reached 92% on WebVoyager with Claude 3.7 (extended thinking). o1 was used as a judge, also we manually double checked the judge. At the core is same old idea - run simple JS script in the browser to identify interactable elements -> draw bounding boxes around them on a screenshot of a browser window -> feed it to the LLM. What made Index so good: 1. We essentially created browser agent observability. We patched Playwright to record the entire browser session while the agent operates,…
- AWA web browser agent in your Chrome side panel2025 · github.com · ▲153
Hey HN, I'm excited to share BrowserBee, a privacy-first AI assistant in your browser that allows you to run and automate tasks using your LLM of choice (currently supports Anthropic, OpenAI, Gemini, and Ollama). Short demo here: https://github.com/user-attachments/assets/209c7042-6d54-4fc... Inspired by projects like Browser Use and Playwright MCP, its main advantage is the browser extension form factor which makes it more convenient for day to day use, especially for less technical users. Its also a bit less cumbersome to use on websites that require you to be…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, January 2025
the whole month →- IM
Hello! I'm Byran. I spent the past ~6 months engineering a laptop from scratch. It's fully open-source on GH at: https://github.com/Hello9999901/laptop
Dev tools · 2025 · byran.ee
- TITetris in a PDF▲1,289
I realized that the PDF engines of modern desktop browsers (PDFium and PDF.js) support JavaScript with enough I/O primitives to make a basic game like Tetris. It was a bit tricky to find a union of features that work in both engines, but in the end it turns out that showing/hiding annotation "fields" works well to make monochrome pixels, and keyboard input can be achieved by typing in a text input box. All in all it's quite janky but a nice reminder of how general purpose PDF scripting can be. The linked PDF is all ASCII so you can just open it in a text editor, or have a look at…
Life & fun · 2025 · th0mas.nl
Create lifelike, personalized AI avatars from text prompts
AI · 2025 · jogg.ai
