The Variation You Know vs. The Variation You Don't

Adopting AI coding assistants does not eliminate variation in software delivery. In most cases I observe, it exchanges a known source of variation for an unknown one. That distinction matters more than it is currently being treated as mattering.

The argument for adoption is not irrational. Human software development has always carried variation. Two engineers given the same task will approach it differently, take different amounts of time, and produce different solutions. That is not a flaw — it is the nature of knowledge work, and I explored it in some detail in the previous article in this series. The point here is that this human variation, for all its frustration, is at least a known quantity. Decades of research in software engineering, organisational psychology, and systems thinking have given us ways to observe it, characterise it, and work with it. We know roughly what a team can deliver. We know how variation tends to behave under pressure. We have built planning and estimation approaches — imperfect ones, but grounded ones — around that understanding.

AI coding assistants — tools like GitHub Copilot, Claude, or Cursor — introduce a different kind of variation. Not smaller. Not larger, necessarily. Different in kind, and critically, as yet uncharacterised.

Think of the long-term effects: do we know of any system created with AI assistants that has been maintained for over five years? The long-term behaviour of AI-generated codebases — how they age, how they accumulate complexity, how they respond to the kind of incremental change that constitutes most of real software maintenance — remains genuinely unknown. We are, as an industry, running an experiment whose results will not be legible for years.

This matters for how we think about variation at the team level. A development team working at a known pace — with understood fluctuation around that pace — represents a system whose behaviour can be modelled, however roughly. When you introduce AI coding assistants without first characterising how they perform in your context, on your codebase, under your constraints, you are not reducing variation. You are exchanging a known source of variation for an unknown one. The distribution of outcomes has not narrowed — it has simply become opaque.

There is a related point worth making about the nature of the assistance itself. The same prompt, submitted to a frontier AI coding assistant, will not reliably produce the same output. The models are not deterministic in the way that a compiler is deterministic. That is an engineering property worth understanding before integrating the tool into a delivery pipeline you are accountable for.

The question worth asking is not whether to adopt AI coding assistants — most teams already have, or will. I am not arguing for slower adoption. I am arguing for informed adoption: understanding what the tool actually does in your specific environment, on your codebase, under your constraints, before the scaling decisions get made. What I find equally worth examining is the mirror image of the certainty trap this series began with. We know that humans prefer the certainty of a known bad outcome over the uncertainty of change. But there is an equal and opposite risk — the moments when uncertainty gets repackaged as certainty, and we adopt it at scale before we have earned the right to that confidence.

If you’re ready to go further, let’s explore how understanding variation in your delivery system can work for your team. Reach out today, and let’s build the evidence base before the scaling decisions get made.