Ourguide – OS wide task guidance system that shows you where to click
Hey! I'm eshaan and I'm building Ourguide -an on-screen task guidance system that can show you where to click step-by-step when you need help. I started building this because whenever I didn’t know how to do something on my computer, I found myself constantly tabbing between chatbots and the app, pasting screenshots, and asking “what do I do next?” Ourguide solves this with two modes. In Guide mode, the app overlays your screen and highlights the specific element to click next, eliminating the need to leave your current window. There is also Ask mode, which is a vision-integrated chat that…
In plain words
Ourguide is an operating system-wide task guidance system that overlays on-screen instructions showing users where to click next. It offers two modes: Guide mode highlights specific elements to follow step-by-step without leaving the current window, while Ask mode provides vision-integrated chat that captures screen context for questions about errors or problems. Built as an Electron app, it works across the entire operating system rather than being limited to browsers, helping users navigate software tasks without constantly switching between applications.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey! I'm eshaan and I'm building Ourguide -an on-screen task guidance system that can show you where to click step-by-step when you need help. I started building this because whenever I didn’t know how to do something on my computer, I found myself constantly tabbing between chatbots and the app, pasting screenshots, and asking “what do I do next?” Ourguide solves this with two modes. In Guide mode, the app overlays your screen and highlights the specific element to click next, eliminating the need to leave your current window. There is also Ask mode, which is a vision-integrated chat that captures your screen context—which you can toggle on and off anytime -so you can ask, "How do I fix this error?" without having to explain what "this" is. It’s an Electron app that works OS-wide, is vision-based, and isn't restricted to the browser. Figuring out how to show the user where to click was the hardest part of the process. I originally trained a computer vision model with 2300 screenshots to identify and segment all UI elements on a screen and used a VLM to find the correct icon to highlight. While this worked extremely well—better than SOTA grounding models like UI Tars—the latency was just too high. I'll be making that CV+VLM pipeline OSS soon, but for now, I’ve resorted to a simpler implementation that achieves <1s latency. You may ask: if I can show you where to click, why can't I just click too? While trying to build computer-use agents during my job in Palo Alto, I hit the core limitation of today’s computer-use models where benchmarks hover in the mid-50% range (OSWorld). VLMs often know what to do but not what it looks like; without reliable visual grounding, agents misclick and stall. So, I built computer use—without the "use." It provides the visual grounding of an agent but keeps the human in the loop for the actual execution to prevent misclicks. I personally use it for the AWS Console's "treasure hunt" UI, like creating a public S3 bucket with specific CORS rules. It’s also been surprisingly helpful for non-technical tasks, like navigating obscure settings in Gradescope or Spotify. Ourguide really works for any task when you’re stuck or don't know what to do. You can download and test Ourguide here: https://ourguide.ai/downloads The project is still very early, and I’d love your feedback on where it fails, where you think it worked well, and which specific niches you think Ourguide would be most helpful for.
More life & fun this month
the category →- TL
Life & fun · 10d ago · louisabraham.github.io
Photosynthesis fires two of your iPhone
Life & fun · 28d ago · photosynthesis.camera
SoloUno▲310Take control of hair pulling, nail biting & skin picking
Life & fun · 28d ago · solouno.io
Scroll through all 43,252,003,274,489,856,000 reachable Rubik's Cube permutations.
Life & fun · 26d ago · everycube.alen.is
Hi HN, I built Eigendrum, a web tool that solves the 2D wave equation for arbitrary shapes so you can hear what they sound like as drums. How it works: * Solves -∇²u = λu using finite element analysis (Kφ = λMφ) on a triangle mesh. * Validated to <0.1% error against closed-form solutions for circles (Bessel zeros) and rectangles. * Sound model factors in strike location, Rayleigh damping, and mallet width. * Includes Kac drums I & II to demonstrate identical sound spectra from different geometries. * No frameworks, build steps, or dependencies. Repo and tests:…
Life & fun · 27d ago · baselashraf81.github.io
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
