Reflect – create end-to-end tests using AI prompts
Hi HN, Three years ago we launched Reflect (https://reflect.run), our no-code end-to-end testing platform, on HN. We're back to show you some new AI-powered features that we believe are a big step forward in the evolution in automated end-to-end testing. Specifically, these features raise the level of abstraction for test creation and maintenance. One of our new AI-powered features is something we call Prompt Steps. Normally in Reflect you create a test by recording your actions as you use your application, but with Prompt steps you define what you want tested by describing it in…
What it does
In the maker’s words, at launch
Hi HN, Three years ago we launched Reflect (https://reflect.run), our no-code end-to-end testing platform, on HN. We're back to show you some new AI-powered features that we believe are a big step forward in the evolution in automated end-to-end testing. Specifically, these features raise the level of abstraction for test creation and maintenance. One of our new AI-powered features is something we call Prompt Steps. Normally in Reflect you create a test by recording your actions as you use your application, but with Prompt steps you define what you want tested by describing it in plain text, and Reflect executes those actions on your behalf. The other feature we're launching is a fallback to using AI to find an element when the selectors we generated are no longer valid. We're making both of these features publicly available so that you can sign up for a free account and try it for yourself. Here's a link to our docs which contains a video demonstrating this feature: https://reflect.run/docs/recording-tests/testing-with-ai/ Our goal with Reflect is to make end-to-end tests fast to create and easy to maintain. A lot of teams face issues with end-to-end tests being flaky and just generally not providing a lot of value. We faced that ourselves at our last startup, and it was the impetus for us to create this product. Since our launch, we've improved the product by making tests execute much faster, reducing VM startup times, adding support for API testing, cross-browser testing etc, and doing a lot of things to reduce flakiness, including some novel stuff like automatically detecting and waiting on asynchronous actions like XHRs and fetches. Although Reflect is used by developers, our primary user is non-technical - someone like a manual tester, or a business analyst at a large company. This means it's important for us to provide ways for these users to express what they want tested without requiring them to write code. We think LLMs can be used to solve some foundational problems these users experience when trying to do automated testing. By letting users express what they want tested in plain English, and having the automation automatically perform those actions, we can provide non-technical users with something very close to the expressivity of code in a workflow that feels very familiar to them. In the testing world there's something called BDD, which stands for Behavior-Driven Development. It's an existing way to express automated tests in plain English. With BDD, a tester or business analyst typically defines how the system should function using an English-language DSL called "Gherkin", and then that specification is turned into an automated test later using a framework called Cucumber. There are two main issues that we've heard a lot when talking to users practicing BDD: 1. They find the Gherkin syntax to be overly restrictive. 2. Because you have to write a whole bunch of code in the DSL translation layer to get the automation to work, non-technical users who are writing the specs have to rely heavily on the developers writing the DSL translation layer. We think our approach solves for these two main issues. Reflect's prompt steps have no predefined DSL. You can write whatever you want, including something that could result in multiple actions (e.g. "Fill out all the form fields with realistic values"). Reflect takes this prompt, analyzes the current state of the DOM, and queries OpenAI to determine what action or set of actions to take to fulfill that instruction. This means that non-technical users who practice BDD can create automated tests without developers having to build any sort of framework under the covers. It's still early days for this technology, but we think our coverage of use cases is wide enough that this is now ready for real-world use. We're excited to launch this publicly, and would love to hear any feedback. Thanks for reading!
Does the same job
all alternatives →
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, June 2023
the whole month →- NI
Heya HN, I've been working on a reddit-like platform as my primary side project for the last few years. Doing a (very) soft launch today, mainly because I want to use it to encourage discussion of alternatives. How non.io works: 1. Free to browse, paid to interact. 2. Minimum subscription is $2 (though you can choose more). I take $1 to run the servers, everything left gets split evenly between everything you upvote that month. It's a simple model, but I hope it's a better one than the freemium model we've been relying on for the last few years. Fundamentally I feel like any ad-supported…
Commerce · 2023 · non.io
Typefully 2.0▲931Effortlessly publish on Twitter and LinkedIn, now with AI
Growth · 2023 · typefully.com
SlashPage▲760Create websites in seconds to build your first 1K community
Growth · 2023 · slashpage.com
