An open-source Operator that can use computers
Hi HF, I'm Terrell, and we built an open-source app that lets developers create their own Operator with a Next.js/React front-end and a flask back-end. The purpose is to simplify spinning up virtual desktops (Xfce, VNC) and automate desktop-based interactions using computer use models like OpenAI’s There are already various cool tools out there that allow you to build your own operator-like experience but they usually only automate web browser actions, or aren’t open sourced/cost a lot to get started. Spongecake allows you to automate desktop-based interactions, and is fully open…
What it does
In the maker’s words, at launch
Hi HF, I'm Terrell, and we built an open-source app that lets developers create their own Operator with a Next.js/React front-end and a flask back-end. The purpose is to simplify spinning up virtual desktops (Xfce, VNC) and automate desktop-based interactions using computer use models like OpenAI’s There are already various cool tools out there that allow you to build your own operator-like experience but they usually only automate web browser actions, or aren’t open sourced/cost a lot to get started. Spongecake allows you to automate desktop-based interactions, and is fully open sourced which will help: - Developers who want to build their own computer use / operator experience - Developers who want to automate workflows in desktop applications with poor / no APIs (super common in industries like supply chain and healthcare) - Developers who want to automate workflows for enterprises with on-prem environments with constraints like VPNs, firewalls, etc (common in healthcare, finance) Technical details: This is technically a web browser pointed at a backend server that 1) manages starting and running pre-configured docker containers, and 2) manages all communication with the computer use agent. [1] is handled by spinning up docker containers with appropriate ports to open up a VNC viewer (so you can view the desktop), an API server (to execute agent commands on the container), a marionette port (to help with scraping web pages), and socat (to help with port forwarding). [2] is handled by sending screenshots from the VM to the computer use agent, and then sending the appropriate actions (e.g., scroll, click) from the agent to the VM using the API server. Some interesting technical challenges we ran into: - Concurrency - We wanted it to be possible to spin up N agents at once to complete tasks in parallel (especially given how slow computer use agents are today). This introduced a ton of complexity with managing ports since the likelihood went up significantly that a port would be taken. - Scrolling issues - The model is really bad at knowing when to scroll, and will scroll a ton on very long pages. To address this, we spun up a Marionette server, and exposed a tool to the agent which will extract a website’s DOM. This way, instead of scrolling all the way to a bottom of a page - the agent can extract the website’s DOM and use that information to find the correct answer What’s next? We're working on adding support in the UI to run this locally on your own machine, and to spin up other desktop environments like Windows and MacOS. We’ve also started working on integrating Anthropic’s computer use model as well. There’s a ton of other features we can build but wanted to put this out there first and see what others would want Would really appreciate your thoughts, and feedback. It's been a blast working on this so far and hope others think it’s as neat as I do :) Here’s the link to clone: https://github.com/aditya-nadkarni/spongecake
Does the same job
all alternatives →
- NONous – Open-Source Agent Framework with Autonomous, SWE Agents, WebUI2024 · github.com · ▲155
Hello HN! The day has finally come to stop adding features and start sharing what I've been building the last 5-6 months. It's a bit of CrewAI, OpenDevon, LangFuse/Cloud all in one, providing devs who prefer TypeScript an integrated framework thats provides a lot out of the box to start experimenting and building agents with. It started after peeking at the LangChain docs a few times and never liking the example code. I began experimenting with automating a simple Jira request from the engineering team to add an index to one of our Google Spanner databases (for context I'm the…
- WBWe Built Open-Source Alternative to OpenAI's Operator, Runs in Chrome2025 · github.com · ▲9
Hi HN, I'm one of the creators of Nanobrowser, an open-source Chrome extension that lets you automate web tasks using AI agents. We were inspired by the potential of tools like OpenAI's Operator, but we wanted something that was: -Open-Source:You can see the code, modify it, and contribute to the project. -Browser-Based:No complex setups or server deployments. It runs directly in your browser. -Customizable:You can tailor the agent's behavior to your specific needs. -BYO LLM:Bring your own large language model API key (OpenAI, Anthropic,or even local models), No vendor lock-in. -Privacy…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, April 2025
the whole month →- IB
Hi everyone, I built PyXL — a hardware processor that executes a custom assembly generated from Python programs, without using a traditional interpreter or virtual machine. It compiles Python -> CPython Bytecode -> Instruction set designed for direct hardware execution. I’m sharing an early benchmark: a GPIO test where PyXL achieves a 480ns round-trip toggle — compared to 14-25 micro seconds on a MicroPython Pyboard - even though PyXL runs at a lower clock (100MHz vs. 168MHz). The design is stack-based, fully pipelined, and preserves Python's dynamic typing without static type restrictions.…
Dev tools · 2025 · runpyxl.com
- UC
Life & fun · 2025 · filiph.github.io
- IB
https://the-pocket.github.io/Tutorial-Codebase-Knowledge/
AI · 2025 · github.com
