How to Write Unit Tests With ChatGPT That Catch Real Bugs

Start with behavior and a function signature

This walkthrough uses a synthetic shipping function. The input is shipping_cost(subtotal: Decimal, member: bool, country: str), and the return value is a Decimal shipping charge. Its approved rules are:

1. Accept the exact country codes US and CA. Raise ValueError for any other code.
2. A member pays no shipping when the subtotal is at least Decimal("50.00").
3. A non-member pays shipping at every subtotal covered by the contract, including 50.00 and above.
4. Paid shipping is Decimal("5.00") for the US and Decimal("8.00") for Canada.

The contract fixes the exact threshold, member qualification, country price, return type, and error for an unsupported code. It says nothing about a negative subtotal, a float, NaN, or an expired membership. Keep those questions out of the tests until someone who owns the behavior answers them. For an existing repository, also check its test runner, Python version, and naming conventions.

Ask ChatGPT for cases before it sees the function body

Use the first prompt to expose omissions while the expected behavior is still independent of implementation branches. OpenAI's ChatGPT prompting guidance recommends clear instructions, enough context, and revising a prompt after reviewing the response.

I need Python unittest cases for this function:
shipping_cost(subtotal: Decimal, member: bool, country: str) -> Decimal

Contract:
- Only exact country codes US and CA are accepted; other codes raise ValueError.
- A member pays Decimal("0.00") at a subtotal of at least Decimal("50.00").
- A non-member always pays shipping within the stated contract.
- Paid shipping is Decimal("5.00") for US and Decimal("8.00") for CA.

Before writing code, make a table with input, expected result, and the
rule each case protects. Cover 49.99, 50.00, and 50.01 for members;
both countries; non-members at and above the threshold; and invalid codes.
Put any input whose expected behavior is unspecified in a separate
"needs a decision" list. Do not infer a rule from implementation branches.

Review the table yourself. In this example, a member at exactly 50.00 must return zero in both countries; a non-member at 100.00 must still pay the country fee. The table should reject MX, lowercase us, and an empty code because the accepted set is exact. It should not assert what a negative subtotal does. If ChatGPT inserts such an assertion, remove it and resolve the policy first.

For your own function, replace the shipping signature, rules, and boundary values with the requirements from your ticket or documentation. Tell ChatGPT which language, test runner, import path, and existing test conventions to use. If you have code but no written behavior, first ask it to list candidate outcomes and unresolved assumptions. Check those against callers, bug reports, and a maintainer before making them assertions; copying the current output can preserve the bug you want a test to catch.

Generate a test file and run it

Once the case table is approved, ask for one file. Keep its expected values tied to the written rules. For a small pure function like this example, the table and signature are enough. In an existing repository, you may also need to give ChatGPT the function body, its dependencies, and one nearby test so it can write correct imports and fixtures. Share those after the expected outcomes are fixed, and tell it to flag any conflict between code and contract instead of copying the current behavior into the assertions. Do not send private repository code to a service your organization has not approved.

Using only the approved case table and contract above, write a complete
test_shipping.py for Python's standard-library unittest.
Import with: from shipping import shipping_cost
Use Decimal literals for money inputs and expected values.
Give boundary and invalid-code tests names that describe the outcome.
Do not add assertions for the "needs a decision" inputs.
Return only the file in one code block. Do not change shipping.py.

If you add existing code and a nearby test to that prompt, add this instruction: “Use the project’s imports and fixtures, but keep the approved expected results. List any behavior the implementation contradicts. Do not edit the function or weaken an assertion to make the new tests pass.”

Save the response as test_shipping.py beside shipping.py. The sample below is a short, editorially reviewed extract showing the assertions that matter most. The downloadable replay script contains a separate 15-method suite covering every row in the table.

import unittest
from decimal import Decimal
from shipping import shipping_cost

class ShippingCostTests(unittest.TestCase):
    def test_us_member_at_exact_threshold_is_free(self):
        self.assertEqual(
            shipping_cost(Decimal("50.00"), True, "US"),
            Decimal("0.00"),
        )

    def test_ca_member_below_threshold_pays_canadian_fee(self):
        self.assertEqual(
            shipping_cost(Decimal("49.99"), True, "CA"),
            Decimal("8.00"),
        )

    def test_ca_nonmember_above_threshold_still_pays(self):
        self.assertEqual(
            shipping_cost(Decimal("100.00"), False, "CA"),
            Decimal("8.00"),
        )

    def test_lowercase_country_is_rejected(self):
        with self.assertRaises(ValueError):
            shipping_cost(Decimal("10.00"), False, "us")

For a first local run, the following reference implementation satisfies only the stated contract. Copy it into shipping.py. It is a teaching fixture, not a recommendation to replace production shipping rules.

from decimal import Decimal

def shipping_cost(subtotal: Decimal, member: bool, country: str) -> Decimal:
    if country not in {"US", "CA"}:
        raise ValueError("unsupported country")
    fee = Decimal("5.00") if country == "US" else Decimal("8.00")
    if member and subtotal >= Decimal("50.00"):
        return Decimal("0.00")
    return fee

Run python3 -m unittest -v test_shipping.py in that directory. The four shown tests should pass with this reference. If discovery says zero tests ran, check the filename, the test_ method names, and the import path before changing any expected value. Python's unittest documentation explains the runner, assertions, and discovery conventions. A green run establishes that these assertions agree with this implementation. It cannot establish that the contract is complete.

Use a deliberate defect to see whether the suite turns red

Change only one behavior at a time. For example, replace >= with > in the local reference, rerun the exact same tests, and look for the exact-threshold member test to fail. Restore the line, then try a version that grants free shipping without checking membership. A red test is useful only when its failure points to the rule you meant to challenge; a syntax error or a broken import does not show that the suite detected a behavior defect.

The download below runs a self-contained 15-method suite against the reference and three controlled variants. It requires only the Python standard library. Save it, then run python3 unit-test-mutation-fixture.py. An exit code of zero means the reference passed and each variant produced exactly the intended failing method names. It does not mean those defects existed in a real checkout system.

In a local Python 3.13.2 run, the reference passed all 15 methods. The wrong-threshold variant failed two exact-threshold member methods; removing the membership condition failed four non-member methods; using the US price for Canada failed four Canadian paid-shipping methods. The runner prints those method names so you can compare its output with the stated risk.

If your own variant leaves the suite green, first confirm that it changes a reachable result under the approved contract. A > threshold bug is visible only at exactly 50.00, so a suite with 49.99 and 50.01 but no exact-boundary case will miss it. A variant that changes only an unapproved input says nothing about the agreed behavior. Likewise, an import error or a syntax error is a broken experiment, not evidence of a strong test suite. Record the changed line, the input that should distinguish it, and the actual failing method before adding more tests.

Handle cases that need a different decision

The first shipping fixture is intentionally small. A useful test plan changes when the starting problem changes. The following situations need different ChatGPT inputs and different checks; use the test runner and language of your own repository.

For that regression scenario, show ChatGPT the written 8.00 Canadian rule, the faulty 5.00 result, and the function signature. Ask for an assertion that shipping_cost(Decimal("100.00"), False, "CA") equals Decimal("8.00"). In the controlled wrong-price variant, that assertion fails because the function returns 5.00. After a fix, rerun it and a US paid-shipping case so the repair does not collapse both prices to 8.00. For a real regression, keep the failing-before and passing-after output with the test.

A legacy implementation may accept a negative subtotal even though no one can find a rule for it. Copying that current result into a new test would preserve accidental behavior. Ask the owner whether the input should raise an error, represent a credit, or be rejected earlier by a caller. Once a decision is written down, give ChatGPT that decision and request the corresponding test. Until then, put the case in the unresolved column rather than inventing an exception.

Worked case: catch an unnecessary service call

Now suppose the shipping rate comes from a service. The unit receives an injected rate_lookup callable: a free member at or above 50.00 must return zero without calling it; a paid shipment must call it once with the country and return that rate; an invalid country must raise before any lookup. The no-call rule matters even when the final shipping charge is correct.

Write test_quote_shipping.py using Python unittest and unittest.mock.Mock.
Function: quote_shipping(subtotal, member, country, rate_lookup)
Import with: from quote_shipping import quote_shipping
Rules: accept only exact US or CA; reject other codes with ValueError
without calling rate_lookup. Members at or above Decimal("50.00")
get Decimal("0.00") without a lookup. Otherwise call rate_lookup
exactly once with the country and return its Decimal result.
Cover a free CA member at 50.00, a paid CA shipment, and an invalid code.
Assert both the returned value and whether the lookup was called.
Use a fake lookup, never a live service. Return one runnable test file.

These are complete teaching files written for this guide, not a retained ChatGPT response. Put them in one directory. quote_shipping.py is the reference implementation:

from decimal import Decimal

def quote_shipping(subtotal: Decimal, member: bool, country: str, rate_lookup) -> Decimal:
    if country not in {"US", "CA"}:
        raise ValueError("unsupported country")
    if member and subtotal >= Decimal("50.00"):
        return Decimal("0.00")
    return rate_lookup(country)

Save the following as test_quote_shipping.py. Python's Mock documentation defines assert_not_called() and assert_called_once_with(), which check the interaction as well as the returned charge.

import unittest
from decimal import Decimal
from unittest.mock import Mock
from quote_shipping import quote_shipping

class QuoteShippingTests(unittest.TestCase):
    def test_free_member_skips_lookup(self):
        lookup = Mock(return_value=Decimal("8.00"))
        self.assertEqual(quote_shipping(Decimal("50.00"), True, "CA", lookup), Decimal("0.00"))
        lookup.assert_not_called()

    def test_paid_canada_looks_up_once(self):
        lookup = Mock(return_value=Decimal("8.00"))
        self.assertEqual(quote_shipping(Decimal("10.00"), False, "CA", lookup), Decimal("8.00"))
        lookup.assert_called_once_with("CA")

    def test_invalid_country_skips_lookup(self):
        lookup = Mock(return_value=Decimal("8.00"))
        with self.assertRaises(ValueError):
            quote_shipping(Decimal("10.00"), False, "MX", lookup)
        lookup.assert_not_called()

Run python3 -m unittest -v test_quote_shipping.py. In a local Python 3.13.2 run, all three tests passed. To challenge them, move the lookup before the free-member check and return the saved rate for paid shipments. The free member still receives 0.00, but test_free_member_skips_lookup fails while the other two tests pass:

AssertionError: Expected 'mock' to not have been called. Called 1 times.
Calls: [call('CA')].

The interaction assertion exposes a defect that a return-value assertion alone would miss. Restore the reference after the check. For a real rate service, keep a separate integration test for request arguments, authentication, and response mapping; this unit test does not exercise the network.

If a date controls eligibility instead, supply a fixed clock or explicit date and test just before, exactly at, and just after the approved cutoff. Those checks should return the same result regardless of when the test runner starts.

Decide what each failure means before changing code

When a generated test fails, read the traceback and compare its expected value with the approved rule. A wrong return value can be a product defect. A wrong expected value is a test defect. ModuleNotFoundError, a missing fixture, or a live API timeout usually calls for an environment or test-boundary repair. If the rule itself does not say what should happen, record a policy gap and ask the owner. Changing production code to satisfy a guessed assertion can make the wrong behavior permanent.

To get help from ChatGPT at this point, give it the rule, failing test, exact run command, and traceback. Ask: “Which line failed, which rule sets the expected value, and is this an implementation defect, a test mistake, or a setup problem? Show the evidence before proposing a change. Do not change the expected value just to make the test pass.” Check its diagnosis against the code and rerun the same test after any fix.

Before committing, check that each retained test names an outcome, uses an expected value independent of production constants, and fails for the intended reason under a relevant defect. Run the repository's full relevant test layer after the focused checks, since a passing unit suite says nothing about another module's integration. The observed results here come from two synthetic Python examples and locally planted defects, not a production checkout. Keep the actual failing input and output beside any regression test you retain.