Reliability

In August 2026 we found 5 benchmark claims on our website that didn’t match the results they were copied from. An audit then compared all 33 numbers in the website’s benchmark data file with their source, and it found that 32 of 33 matched exactly. The one that didn’t was the only field whose source had changed since someone copied it. A citation tells you where a number came from. It doesn’t tell you whether the number still matches the source, so the website and this blog now check that in CI. ...

If your automation counts a run as a success when it reaches its last step without an error, that count includes the runs that saved the wrong thing. In our own fault test, a check that trusted the success banner passed 54 of 72 bad saves. A write audit separates those runs from the good ones. You compare what each run saved with the record itself, and you report two numbers: the bad saves that got through, and the good saves your automation stopped anyway. The method works on a recorded RPA process, a Playwright script, a Power Automate desktop flow, a computer-use agent, or a person following a checklist. You can run it with tools you already have. ...