In simple English
Writing tests for software usually breaks every time a button moves or changes its name. This tool lets you write tests in plain sentences instead of fragile code paths. An artificial intelligence looks at your screen like a human does to find buttons and check results. It works across web browsers, mobile phones, and desktop computers. You will need to pay an external model provider for the vision capabilities used during test runs.
What it is
Midscene is a testing framework that uses visual AI agents to interact with user interfaces through natural language instructions. It supports integration with Playwright (a tool for automating web browsers from code) and YAML (a simple file format for writing configuration and data) test files.
What it is good for
Solo builders who want to write end-to-end tests quickly without spending hours fixing broken locators every time their user interface changes.
- Tests do not break when internal HTML selectors change.
- Works across web, iOS, Android, and desktop applications.
- Generates clear visual HTML reports with screenshots of every step.
- Requires an active API key and ongoing budget for a vision language model.
- Test execution is slower than traditional code-based locators.
- Debugging unexpected AI agent failures can be difficult.
Traditional brittle end-to-end test suites using XPath or CSS selectors.
Playwright (a library to automate browser actions from code).
- microsoft/playwright98k stars
Automates browsers with code, but requires manual element selectors instead of visual AI instructions.
- puppeteer/puppeteer96k stars
Controls browsers using JavaScript, but lacks built-in natural language assertions and visual agents.
- unocss/unocss19k stars
A styling engine that reads the labels in your markup and generates only the CSS those labels need.
- HypothesisWorks/hypothesis9.1k stars
A Python library that generates test inputs and reduces any failure to the smallest example that still fails.
npm install @midscene/web