How to review AI-assisted code before it ships

A coding agent finishes a change in twenty minutes. The diff looks clean, includes tests and docs, and passes the pipeline. That presentation can make the change feel more thoroughly understood than it is: the implementation, tests, and summary may all repeat the same mistaken assumption because they were produced from the same prompt and repository context.

Now ask the engineer who owns it three questions:

1. Which assumption would cause the most damage if it is wrong?
2. What evidence came from outside the implementation’s own reasoning?
3. How would you diagnose and reverse this change at 2 a.m.?

If the answers are vague, the work is not ready. More generated documentation will only restate the change; it will not supply independent evidence, an operational mental model, or a person prepared to own the consequences.

The main risk in AI-assisted development is not that an agent sometimes writes bad code. People do that too. The risk is that finished-looking output can grow faster than a team’s ability to understand, verify, and operate it.

The useful response is a stronger version of the review process good teams already use: define the outcome, limit access, verify behavior independently, and make sure an engineer can own the result. AI changes the speed and volume of the work. It does not require new names for those responsibilities.

Reviewing an AI-assisted change

Open in Studio

Define the outcome before generating code

Agents are good at turning a request into artifacts. That makes a loose request dangerous. “Add role management” can become a new permission model, migration framework, compatibility layer, and admin API before the team has agreed on who may grant which role.

Start with a bounded outcome and the constraints that matter:

• Who needs the change, and what should become possible?
• Which behavior must not change?
• Which data, trust, and architectural boundaries are involved?
• What failure would be expensive or hard to reverse?
• What is explicitly outside this change?

For a low-risk example, imagine adding CSV export to an internal report. The outcome might be:

An authorized analyst can export the filtered rows already visible on screen. The export must use the same authorization and tenant filters as the report. It must not introduce a second query path.

That statement gives the agent room to help without letting it quietly redesign reporting or access control.

I also ask one deletion question during review: What could we remove and still meet the outcome? Cheap code is not cheap to explain, patch, secure, and eventually retire.

Limit access instead of trusting the prompt

An agent that can read a repository, run commands, install packages, call services, or deploy code is an actor with capabilities. A sentence that says “do not access production” is guidance, not a security boundary.

Use the controls we already understand:

• isolate the working environment
• grant the least privilege needed for the task
• keep production credentials and customer data out of reach
• separate permission to write code from permission to merge or deploy it
• restrict tools, destinations, and dependencies when the risk calls for it
• log consequential actions and make access easy to revoke
• keep a person in control of irreversible or high-impact operations

For the CSV export, the agent needs the repository and test environment. It does not need production data, deployment credentials, or access to the analytics warehouse. Those limits should come from the environment, not from trust in a prompt.

NIST’s Generative AI Profile places generative AI inside ordinary risk management. Identity, authorization, audit, and incident response still apply.

Verify behavior independently

An agent can write the feature, derive tests from its own code, summarize the passing results, and draft the review description. The artifacts agree because they may share the same mistaken assumption.

Tests written with agent help are still useful. They just should not be the only anchor for an important claim.

For the export feature, write acceptance examples before implementation:

• an analyst exports only the rows allowed by the current tenant and filters
• a user without report access receives no file
• a spreadsheet formula in a cell is escaped rather than executed when the file opens
• an export that exceeds the size limit fails without exhausting application memory

Then verify the result through the public behavior. Do not inspect the generated function, mirror its branches in a test, and call that independent proof.

This distinction matters because perceived speed and measured speed can disagree. In a 2025 randomized study, METR found experienced open-source developers took 19 percent longer on the studied tasks with early-2025 AI tools, even though participants believed the tools made them faster. That is one setting, not a universal verdict. It is a good reason to measure outcomes instead of treating fluent output as proof of productivity.

Check that a person can own the change

An approval records a decision. It does not prove that the reviewer or author has a useful mental model.

For a consequential change, ask the accountable engineer to walk a peer through it without relying on the agent transcript:

• What outcome does the change produce?
• What are its important assumptions?
• Which boundaries does it cross?
• How does it fail, and how will the team notice?
• What do the tests prove, and what do they not prove?
• How can the team disable, reverse, or repair it?

This is not an oral exam. If an engineer cannot explain the change, the right response is time, pairing, and a smaller change. Shame produces ceremonial approval, not understanding.

The last ownership test happens after merge. Can someone other than the original author change the feature, diagnose a staged failure, and remove it if it is no longer needed? If not, the team has created dependency, even if that dependency is now a tool instead of one senior engineer.

Run a bounded pilot before changing the whole system

Do not begin with an organization-wide productivity target. Pick one workflow with a real outcome, manageable failure cost, and a baseline you can measure.

For example, pilot agent assistance on small API maintenance changes for thirty days. Before the pilot, record normal lead time, review time, escaped defects, rollback rate, and how confident the team feels modifying the area. During the pilot, keep change size and release controls stable. At the end, compare outcomes and hold a retrospective with developers, reviewers, QA, security, and operations.

The decision is not simply “Was AI faster?” Ask:

• Did verified changes reach users sooner?
• Did review or support work move somewhere less visible?
• Did defects, security findings, or rollbacks change?
• Can more than one engineer explain and modify the result?
• Which constraints helped, and which were ceremony?

DORA’s 2025 research on AI-assisted software development describes AI as an amplifier of the engineering system around it. Strong feedback loops and clear platforms help teams use the leverage well. Weak systems can produce weak results faster.

Ship only what the team can support

I think about engineering as stewardship. Capability should serve people, and speed does not excuse us from understanding the consequences of our work. That belief is personal, but the engineering standard is available to any team: preserve human agency, seek independent evidence, admit what is not understood, and stay accountable to the people who live with the software.

Use agents aggressively where work is bounded, reversible, observable, and easy to verify. Add human judgment where consequence, ambiguity, and recovery demand it.

Move quickly, but do not ship faster than the team can understand what it is taking responsibility for.