How to use AI to propose tests that find bugs
Using AI to write tests works better when you present the rule before the code. The tool’s job is to suggest ways to challenge behavior, while you decide which results are correct.
A large collection of checks may repeat the same idea. In this tutorial, we will choose a few cases that distinguish a correct implementation from a plausible mistake.
1. Turn the rule into a checkable statement
Imagine a function indicating whether an integer age permits participation in an activity: people aged 18 or over are eligible. In this teaching example, the input domain is already restricted to nonnegative integer ages.
That definition keeps the main rule separate from form handling or text conversion. If those decisions also belong to the project, they need separate requirements.
2. Ask for cases before test code
Request a table containing the input, expected output and reason for each case. This makes an incorrect interpretation easier to spot before generating files.
Rule: nonnegative integer ages are eligible from age 18.
Propose a few cases that detect different mistakes.
For each case, give the input, result and mistake it finds.
Do not use the implementation as the source of expectations.
Identify ambiguities before writing automated tests.In this example, 17, 18 and 19 serve distinct purposes. The value exactly at the threshold matters because it distinguishes “18 or over” from “over 18.”
3. Select a focused set
Review the proposed table and keep cases with a clear justification. Including several ages above 18 is not automatically better than covering the correct boundary.
| Age | Expected | Risk observed |
|---|---|---|
| 17 | Not eligible | Accepting before the threshold |
| 18 | Eligible | Excluding the boundary |
| 19 | Eligible | Rejecting beyond the boundary |
If an input’s behavior is undefined, do not let AI silently invent it. Resolve the rule before turning an assumption into a permanent test.
4. Write readable checks
A simple representation helps you review the outcome:
assert elegivel(17) is False
assert elegivel(18) is True
assert elegivel(19) is TrueThe pytest assertions guide shows how expectations are expressed in Python tests. A test tool organizes execution; it does not choose business rules for you.
5. Check whether a mistake would be noticed
In an isolated copy of the example, temporarily change idade >= 18 to idade > 18. The check for 18 should fail.
Then restore the correct version and run it again. If the test still passes with the wrong rule, investigate whether it calls the correct function and compares the result.
6. Review the diagnosis, not only the result color
When a test fails, read the input, expected output and actual output. A configuration failure does not prove the behavior was exercised.
Ask AI to explain the mistake found and the coverage still missing. To check a proposed implementation, also follow the first-test guide.
Frequently asked questions
What makes an AI-suggested test useful?
It checks required behavior and fails when that behavior is wrong. The expectation should come from the requirement rather than the implementation.
Do I need to accept every test AI suggests?
No. Select tests covering different risks and discard repetition or unnecessary checks of internal details.
How can I check that a test detects a bug?
In an isolated example, introduce a small change that breaks the rule and check that the test fails. Then restore the correct version.