AI
Work2Skill turns human GUI demonstrations into reusable agent procedures
Work2Skill proposes capturing synchronized screen recordings, mouse and keyboard events, and accessibility-tree data so agents can extract reusable workflow skills from human demonstrations. Its WorkDemo-350 dataset contains 350 screened, successful desktop tasks. The authors report that skills induced by their method improved scores over no-skill baselines across MyPCBench and a 286-task OSWorld-Verified subset for four execution models. The evidence remains an initial research evaluation, not proof that the approach generalizes to arbitrary software or workflows.
Editorial summary · Feed excerpt
What matters
- The dataset aligns multimodal GUI evidence with human actions and task outcomes.
- Reported mean gains range from 4.67 to 10.54 points on MyPCBench and 3.11 to 7.50 points on the OSWorld subset.
- The method attempts to separate general workflow rules from task-specific details and includes completion checks.
Context & caveats
- The supplied material does not identify the benchmark scales or provide experiment details beyond the abstract.
Why this made the edition
The abstract describes a concrete data-collection system, dataset and cross-benchmark comparison with quantified gains, making it useful for readers tracking computer-use agents.
Source material