Computer-Use

Computer-use agents are good now. Point one at a screenshot and a goal and it’ll find the button, on software it has never seen, with no selectors and no API. That still feels a little like magic to me. But watch one do the same task for the fiftieth time. It re-reasons everything, every run. Same login screen, same referral queue, same eleven clicks. Each time, a frontier model stares at the pixels and thinks the whole thing through again, and you pay for that thinking in seconds and tokens. Occasionally you also pay in a creative wrong answer. ...

The obvious objection to the 500th run was “sure, it’s your demo app.” Fair. So on 2026-07-08 we ran the same head-to-head against the official OpenEMR public demo: a dense, frame-heavy, LAMP-era EMR that anyone can point software at, fake patients only. Quick recap of what’s being compared. openadapt-flow compiles a recorded demonstration (browser, desktop, Citrix) into a deterministic workflow. Every step carries redundant visual evidence (a template crop, an OCR label, geometry landmarks) plus postconditions derived from what your demonstration actually changed on screen. A healthy run makes zero model calls. When the UI drifts, a resolution ladder heals the step and writes the fix back as a reviewable diff. When verification fails, it halts. It doesn’t improvise against a patient chart. ...