Verifying AI Output

Why it needs checking

Assistants predict likely text, and likely text is not always true. Typical failures include:

• Calls to functions, parameters or packages that do not exist, or that exist only in another version.
• Code that works for the happy path but ignores empty input, null values, large inputs, time zones or concurrency.
• Insecure patterns, such as building SQL from strings, weak validation or hard-coded secrets. See Common Vulnerabilities and Input Validation.
• Code that solves a slightly different problem from the one you asked about, or ignores your project’s conventions.
• Out-of-date advice, and code that resembles licensed material; see Licensing and Open Source.

A verification routine

1. Read it all. Do not paste what you do not understand. Ask the assistant to explain it, then check the explanation against the code and the documentation.
2. Check it against the official docs. Confirm that each API, flag and package exists and behaves as claimed. Verify that a suggested dependency is real and maintained; see Dependency Risks.
3. Run it. Try it with typical input, then with awkward input: empty, huge, malformed, unexpected characters.
4. Test it. Write or review unit tests that state the expected behaviour independently of the generated code, including edge cases and error paths. Better still, write the tests first, as the specification, and let the assistant try to satisfy them. Generated tests need the same scrutiny: check that they can fail, and that they assert something meaningful instead of mirroring the code.
5. Review it as you would a colleague’s change. Keep it small, readable and consistent with the codebase; see Pull Requests and Code Reviews and Clean Code and Naming.
6. Check security and data handling. Look at inputs, outputs, error messages, logging of sensitive data and secrets.

Scale checking to risk

A throwaway script needs less than code that handles money, personal data or authentication. The less you understand a piece of code, the more verification it needs; if you cannot verify it, do not use it.

Keep ownership

If you cannot explain it, change it or debug it later, you have borrowed a problem. Do not ask a reviewer to find the mistakes you did not look for. Mention in the pull request when a significant part was generated, if your team expects that.

Try it

Ask an approved assistant for a small function with some edge cases, such as splitting a string into chunks of a given size, or checking whether a year is a leap year. Before reading its answer carefully, write five unit tests of your own, including empty input and boundary values. Run them against the generated code and fix or reject whatever fails.