How to Review Code With ChatGPT Before You Ship

Give ChatGPT the code and its contract

Start with one changed behavior, not a request to “find bugs.” A useful packet gives ChatGPT enough information to say what should happen, what the candidate code does, and which facts are still missing. For a pull request, include the changed lines and the relevant caller or interface; a lone function may hide where an identity, path, or database connection comes from. If the change touches unrelated behaviors, make a separate packet for each one.

Remove credentials and private records before sharing code, and use an environment your team has approved. Do not silently fill gaps with guesses: if the schema or authentication path is unavailable, name it as unknown. The packet below is the exact synthetic order lookup sent in our August 19 ChatGPT run. The signed-in user's ID is separate from the order ID in a URL; the function promises to return an order only to its owner. The prompt includes the vulnerable code, so it can be copied as written.

Review this synthetic Python change before release. Intended behavior: return an order only when it belongs to the signed-in user. order_id is an untrusted string from a URL parameter. user_id is the authenticated integer user ID. The connection exposes DB-API-style execute(sql, params) and fetchone().

Return:
1. release-blocking findings ordered by severity, with a concrete reproduction;
2. a minimal patched function;
3. regression tests or test cases for every blocking finding;
4. non-blocking limits that require repository context.

Do not invent surrounding middleware, schema constraints, or framework behavior. Do not recommend string escaping as a substitute for parameter binding.

def get_order(conn, user_id: int, order_id: str):
    """Return an order only when it belongs to the signed-in user."""
    sql = (
        "SELECT id, user_id, total_cents, status "
        f"FROM orders WHERE id = {order_id}"
    )
    row = conn.execute(sql).fetchone()
    if row is None:
        return None
    return dict(row)

The example contains no real credentials, users, or database records.

Before sending the prompt, we recorded two expected release blockers: the URL value is inserted into SQL text, and user_id is never used. We also recorded what a useful answer needed to show: an input that changes SQL behavior, a cross-user lookup, a minimal fix, and tests tied to those failures. This ledger helps distinguish a real finding from an alarming label. It does not prove there are only two defects.

Reproduce the recorded security findings

ChatGPT called the SQL interpolation critical and the missing ownership check high severity. For the first finding it supplied order_id="1 OR 1=1". The original function builds WHERE id = 1 OR 1=1, so the input changes the query's meaning. That establishes a reachable query defect in this fixture; which row is returned depends on the data and query plan. For the second, user 100 can ask for order 42 belonging to user 200 because the query never uses user_id. This disclosure needs no injection string. The two findings require different checks and both matched the prewritten ledger. The severity labels came from this synthetic review; a real team's release priority depends on the exposed route and data.

OWASP's SQL injection guidance recommends parameterized queries so input remains data. Escaping a string by hand would leave the underlying query construction fragile. OWASP's authorization guidance treats access checks on a specific object as a separate concern. Binding order_id alone would close the injection path but still allow user 100 to fetch user 200's order.

Inspect the patch, including its assumptions

ChatGPT proposed the function below. Compared with the original, it replaces interpolation with two bound values and adds the owner predicate. It leaves the return shape alone.

def get_order(conn, user_id: int, order_id: str):
    """Return an order only when it belongs to the signed-in user."""
    sql = (
        "SELECT id, user_id, total_cents, status "
        "FROM orders WHERE id = ? AND user_id = ?"
    )
    row = conn.execute(sql, (order_id, user_id)).fetchone()
    if row is None:
        return None
    return dict(row)

The ? placeholders work with Python's sqlite3 module, which documents both value binding and row factories. A production driver may use another placeholder style. The owner predicate also helps only if the caller supplies user_id from the authenticated session; accepting that ID from the request would let the requester choose whose order to look up. Trace the value through the route before marking the authorization finding fixed.

The patch preserves dict(row) from the original. In a separate local probe, the fixed lookup returned a dictionary when SQLite used sqlite3.Row, but raised TypeError on an owned row with SQLite's default tuple row. That is an existing interface assumption, not a new bug introduced by the patch. Confirm the application's row adapter rather than changing the return type to make this example pass. Likewise, the query treats a malformed ID as a value that finds no row; whether the HTTP route should return “not found” or a validation error is a product rule the prompt did not supply.

Run one contract against the old and fixed code

The August 19 ChatGPT response included tests with a fake connection. Four copied tests passed under Python 3.13.2, but the fake returned a preselected row regardless of SQL. Its “non-owned order” test therefore could not prove database ownership filtering. Calling the old function through that same fake fails first because the fake requires a second execute() argument. That interface error is not a useful reproduction of either security defect.

On September 27 we added a separate local exercise using an in-memory SQLite database. It creates order 1 for user 100 and order 42 for user 200, sets sqlite3.Row so dict(row) works, and applies the same four behavior checks to both implementations. Download the complete Python practice kit for the original function, the patch, the recorded prompt, and the test file.

REVIEW_IMPL=unsafe python3 -m unittest -v test_order_lookup
python3 -m unittest -v test_order_lookup

The first command is supposed to exit with two failures. It exposes what the old code does; it is not a successful test run. The second command selects the fixed function and should pass all four cases.

Our retained run produced exactly those results: two failures on the unsafe function, then four passes on the fixed function under Python 3.13.2. Keep the same input rows and assertions when switching implementations. If the “old” run fails only because a mock has the wrong method signature, it has not reproduced the reported vulnerability. If both runs pass, the test may not reach the changed behavior. If both fail, inspect the fixture and the contract before treating the patch as wrong.

The SQLite exercise checks database behavior more directly than the fake connection, yet it cannot certify another driver or the application's request path. In a real service, create two test accounts and two orders in its test database. Call the endpoint as user 100 for user 200's order, then try an injection-shaped ID through the same route. Check both the response and whether the handler derived user_id from authentication. Run the repository's broader test suite after the narrow regression; a local fix can disturb other callers.

Review a separate correctness regression

A review should also catch a change that passes security checks but returns the wrong result. This separate editorial fixture uses a one-based page number: page 1 starts at row offset 0, page 2 at offset 20 when the page size is 20. A candidate change replaces (page - 1) * size with page * size while leaving validation in place:

def page_offset(page: int, size: int) -> int:
    if page < 1 or size < 1:
        raise ValueError("page and size must be positive")
    return page * size  # candidate change; violates one-based contract

Give ChatGPT the page-numbering rule, the changed line, and two expected outputs. In the practice kit, the candidate fails the first- and second-page checks; replacing the return expression with (page - 1) * size passes all four. The invalid-page and invalid-size checks pass in both versions, which shows that the regression is in offset calculation rather than input validation.

PAGINATION_IMPL=candidate python3 -m unittest -v test_pagination
python3 -m unittest -v test_pagination

If the real API specifies zero-based pages, page * size may be the right offset formula, but this candidate would still reject page 0. The contract and validation would both need review. If the API uses cursors, this offset calculation may not apply at all. Read the caller's contract before accepting a confident off-by-one diagnosis. This pagination review and its tests were written later by the editorial team; they were not part of the August ChatGPT run.

Investigate a plausible finding before ranking it

Some comments cannot be settled from the function alone. Imagine a reviewer suggests adding LIMIT 1 to the order query to prevent duplicate results. In the kit's SQLite schema, id INTEGER PRIMARY KEY makes that ID unique, so LIMIT 1 would not correct this lookup's behavior. If the real query instead uses an order number that can repeat across tenants, the schema and tenant predicate matter; adding a limit would merely hide which row won. This is an editorial counterexample, not a claim from the recorded ChatGPT response.

Ask the model to identify the missing fact rather than turn a guess into a blocker. A useful follow-up is: “Your finding assumes this lookup can return more than one row. The test schema declares id as the primary key. Re-evaluate the claim against that schema. State what migration or production difference would change your answer.” The maintainer still checks the actual schema. A model's retraction is not evidence that the database matches the test fixture.

Make the release decision

Record the result in terms another maintainer can verify. For the order fixture, that record looks like this:

Hold the release if the cross-user lookup works, an input still changes SQL syntax, or a required integration check has not run. If a new model finding lacks a reproduction, investigate the exact assumption and keep the release on hold while a credible safety or correctness risk is unresolved. If the risk is disproved by the actual schema or caller, record that evidence and move on. GitHub's review guidance likewise calls for functional checks, context, and human review of AI-assisted code. The maintainer uses the final diff and application checks to make the release decision.

When a review reveals a separate observed failure, follow the debugging workflow. For deeper coverage design, use the behavior-first unit-test guide; the practice kit here is meant to check a specific release change.