Running

Self-repair

When the app changed under a flow, the edit is proposed on the pull request — never applied.

Most test failures after a UI change are not bugs: a button was renamed, a screen reworded, and the flow still says the old words. TestedOK notices, writes the edit, and proposes it where the change is being reviewed — as a comment on the pull request. It never edits a flow itself.

What happens

When a flow fails (not errors), TestedOK looks at the flow, the reason, the screen it failed on, and — when there is one — the screen the last passing run saw at the same step, and decides:

  • the app changed — a control the flow names is gone and something close is there ("Add to cart" → "Add to bag"), the expected text was reworded, a screen was renamed. The flow should follow the app.
  • a bug — an error state, a missing feature, data that should be there. The flow is right.
  • unsure — the screens do not say.

For the app changed it writes the smallest edit that makes the flow describe the app as it now is: the renamed controls, quoted exactly as the screen labels them; the expected text as it now reads. It never loosens an expectation to make a failing flow pass, and never names a control that is not on the screen.

Where it shows

  • On the run page, for every failed run with a proposal: the verdict, why, and the edit as a diff.
  • On the pull request, when the run came from one and the GitHub App is connected and the verdict is the app changed: one comment per flow and commit, with the same diff. Apply it in the flow file if the change is intended; if it is not, the failure is a bug.
-  "goal": "On the \"Cart\" tab, tap \"Add to cart\", then \"Checkout\"."
+  "goal": "On the \"Cart\" tab, tap \"Add to bag\", then \"Check out\"."

Without a model

The comparison alone — names the flow uses that the failing screen no longer has, and the nearest label that is there — needs no model and always runs. The judgement between changed and broken uses the same model as escalation; without it, a proposal is shown on the run page as unsure and no comment is posted.

Limits

It reads labels, not layouts: a control that moved to another screen, or a flow that now needs an extra step, gets a proposal only when the words say so. A flow whose wording was wrong from the start is treated as the app changed with the right name, which is the same fix.