Run openadapt quickstart --break-it and you’ll watch a fake clinic app with synthetic patients report a save that its server rejected. OpenAdapt reads the record through a separate interface and finds no saved note, so it stops at the Save step and asks a person to check the record. The demo takes a few minutes and keeps its files on your computer. Run it You need Python 3.10, 3.11, or 3.12, which is what openadapt 1.16.0 supports. ...
If your automation counts a run as a success when it reaches its last step without an error, that count includes the runs that saved the wrong thing. In our own fault test, a check that trusted the success banner passed 54 of 72 bad saves. A write audit separates those runs from the good ones. You compare what each run saved with the record itself, and you report two numbers: the bad saves that got through, and the good saves your automation stopped anyway. The method works on a recorded RPA process, a Playwright script, a Power Automate desktop flow, a computer-use agent, or a person following a checklist. You can run it with tools you already have. ...
Most automation decides that a save worked by looking for the green “Saved” banner. We tested how far that banner can be trusted. In our fault test, 72 of 90 runs left the record wrong. A check that trusted the banner passed 54 of 72 bad saves (75.0%). A check that read the saved record back by a separate route passed 9 of 72 bad saves (12.5%). A bad save that reports success does more harm than a crash, because nobody gets paged. The error turns up later with whoever owns the record, as a note in the wrong patient’s chart or a payment on the wrong loan. The automation’s check confirmed that something was saved, without asking whose record it was. ...
On the public OpenEMR demo, a compiled replay finished 19 of 20 runs of an 18-step charting task and made no model calls. A computer-use agent finished 10 of 10 and spent about $0.55 a run on model fees. The agent had the higher success rate. On the one compiled miss, the program clicked Save, saw that the screen didn’t change as recorded, and stopped instead of reporting success. Compiled replay was also 1.8x faster at the median. ...