This loop does not trust a single green run: it requires a streak of clean passes to prove stability.
The cycle runs the realistic scenario suite, fixes every failure (including flaky ones), resets the counter on any failure, and continues.
It stops only when the suite passes N times in a row (5 by default) with zero failures.
That way it hunts down flakiness and intermittent bugs a single run would hide.
When to use
- When you suspect flaky tests or intermittent bugs.
- Before a release that needs real confidence in the suite.
- To tell a real bug apart from plain flakiness.
How to use
Run the loop asking it to run the scenario suite, fix any failure, reset the streak on each failure, and continue until it passes N times in a row.