midscene-js

#midscene-js

@quanruzhuoxiu: My favorite design in Midscene.js is actually not the AI part, but the HTML replay report. Every time a script runs, it automatically generates a single-file HTML report containing: - Screenshots of each step - Full prompt input to the model - JSON output from the model (...

X AI KOLs Timeline ↗ · 2026-05-22 Cached

Midscene.js's HTML replay report design helps developers quickly locate the cause of AI automation failures through the triple combination of screenshots, prompt, and model output.

0 favorites 0 likes

#midscene-js

@quanruzhuoxiu: Over the two years of developing Midscene.js, we made a belated but critical decision: UI automation will sooner or later shift from 'understanding the DOM' to 'looking at the screen'. So in the December 1.0 release, we directly cut the DOM compatibility path. In the early days, like everyone else, we followed a DOM + visual hybrid approach...

X AI KOLs Timeline ↗ · 2026-05-14

The Midscene.js team decided to completely shift from a DOM + visual hybrid approach to pure visual UI automation, believing that future UI automation must be based on screenshots rather than the DOM. This change reduced token consumption and simplified cross-platform adaptation.

0 favorites 0 likes

midscene-js

@quanruzhuoxiu: My favorite design in Midscene.js is actually not the AI part, but the HTML replay report. Every time a script runs, it automatically generates a single-file HTML report containing: - Screenshots of each step - Full prompt input to the model - JSON output from the model (...

Submit Feedback