The fake double diamond is dead. The real double diamond is immortal.

AI is not a substitute for divergent thinking, and the false certainty it gives will never save you from having to iterate.

The discourse loves a dichotomy. Framing any disagreement as a difference of opinion between two sides, and then asking everyone to pick a camp, is intellectually convenient. It also happens to be a really bad way of actually wrapping your head around the possibilities present in the world.

This tendency manifests itself in lots of different ways, but this is a product blog, so we are going to talk about product development stuff. Specifically, how orgs learn. For which there are (of course) two main camps: the Agile way (where we build to learn); and the Waterfall way — usually defined simply as the opposite of the Agile way, because it’s less of a methodology and more of a rhetorical device. The Waterfall way of “big design up front” is implied to be practiced by some other people, outside the room. Inside the room are only innovators, who are above that kind of thing.

At least, that’s what we say. But what we actually do couldn’t be further from the truth:

Organizations often say they want innovation, but at the precise moment when they need exploration most, they often seek out predictability.

This is why we are still doing Agile transformation 25 years after everyone agreed Agile was the right way to build stuff. The dichotomy between up-front definition and iteration was always wrong and trying to stamp out one tendency in service of the other will continue to fail, no matter how much AI we pour onto the problem.

The eternal quest to get BDUF for free

No amount of manifestos could suppress the simple human tendency that certainty feels good. To the people in charge of calendars, there is nothing more alluring than the idea that you can simply know the right thing to do up front, and then do that thing exactly once, never having to budget time for testing or rework.

The motivating factor for perfect up-front learning is not “doing a good job.” It’s actually exactly the same motivation that got orgs to pay lip service to Agile in the first place: spending less time before teams can start producing outputs. That Waterfall-ish desire for data up front is not in contrast to “build to learn” but another way of arriving at the same goal: permission to ship the thing we wanted to ship in the first place.

Is the biggest barrier to evidence-based product decisions or policy a lack of evidence? Not usually.

This is the world in which synthetic users are considered to be an alternative to real research; not because they are good but because they remove the friction of challenging your existing opinions. Their dismal accuracy (you’re better off flipping a coin) is besides the point, because the product being sold is not real insights. They are worse than simply having nothing, because when you have nothing then you at least know what you don’t know.

The pursuit of velocity has led many to argue passionately that the “discovery phase” should be abolished. I think that is completely the wrong lesson to draw from the industry. It’s not that up-front discovery is wrong, it’s that it’s insufficient by its lonesome. To be effective, it must work in concert with the rest of the process:

Just like an airplane’s time on the runway is still part of the flight, discovery is part of design and must give creative work momentum and direction.

This is where synthetic insights will always fail. Because they are an excuse not to do discovery (rather than a real discovery activity) they are incongruent with the creative process. When the answer is “AI told me to,” one cannot give a compelling explanation for choosing one direction over another — at least not one that is immune to being overturned by a stakeholder telling the same AI to give a different answer.

Only a human expert — bolstered by both an understanding of the data and the confidence drawn from professional experience — can prevent this process from being perverted into going through the motions and defaulting to the thing the stakeholders wanted in the first place. Which brings us back to Strawbridge’s piece from the beginning of the issue:

Innovation requires more than exploration. It also requires expertise. It benefits from a seasoned guide. […] Senior people can … resist the urge to force a problem into a predetermined framework, follow something unexpected when it emerges, and stay with a question a little longer before converging on an answer.

Up-front certainty was always a fool’s errand, at whatever cost. To try and attain it at bargain-basement prices is not a “hack” that your competitors have yet to discover. It’s simply doubling down on the foolishness.

Prototype ≠ experiment

Ultimately, an ersatz discovery phase (either one driven by synthetic user research, or one focused on “validating” what executives already believe) gets you to the same place as not doing discovery at all: writing the code you wanted to write from the beginning. This, some people suggest, is a reason to avoid doing discovery entirely.

And I don’t just mean up-front discovery. There is an increasingly popular notion that because code is cheap now (hah!) you can simply ship whatever you want and let analytics sort the wheat from the chaff. This is, of course, a radical misunderstanding of the nature of wicked problems. Attempting to solve the problem changes the problem; without giving deliberate thought to the lifecycle of your experiments (including the clean-up) you are merely polluting your product and praying that users won’t notice.

A real experiment is not just a shipped feature that was poorly thought out by its designers. An experiment is a deliberate task that a team undertakes to learn something it does not already know. It requires a hypothesis — and “users will like it” (or worse, “stakeholders will like it”) is not a real hypothesis. And once that thing is learned, the experiment comes to an end.

Teams whose so-called experiments are launched without that arc in mind are not running experiments at all; they do not know what they are looking to learn, how the experiment will help them learn that thing, or even when that will come to pass. They cannot reverse the experiments where the hypothesis was disproven. They can only double down on experience rot.

The problem is that many organisations talk as though they are playing poker while building products as though they are playing chess. They describe decisions as experiments, but rarely remove the failed ones. They promise to iterate, but almost never return. Each supposedly temporary bet leaves another permanent mark on the board.

The confounding factor is that for most orgs, this feels like it works. Things ship, middle managers are happy — and yet nothing really gets better. The simplistic notion of “this feature will make that number goes up” is ground underfoot by the complexity of socio-technical systems. The impacts of any change are inherently delayed, difficult to observe and isolate any attributable cause unless you know exactly what you are looking for.

Making your systems work for you

It’s easy to take one side or the other in this dilemma. If you favor iterative development, then there are many examples of discovery done badly to point at and say “we can’t do it that way.” But at the same time, if you are coming from the other side, it’s equally simple to point at any number of terrible products produced by teams who could not iterate their way away from their thoughtless starting point.

The argument between “up-front research” and “ongoing research” is a rhetorical trick that only serves people who care for neither. Those people come to the meeting with a quiver of excuses for why we mustn’t take any time to think before shipping the thing, and yet simultaneously can’t look back at what we’ve shipped to see if it worked because it’s time to ship the next thing.

This is the fake double diamond: substituting real divergence and convergence with artificial alignment and a synthetic sense of urgency, whipping the team ever-forward at a pace that ensures the appearance of brilliance as no decision can ever actually be revisited.

For those of us interested in actually making useful software, we can’t get rid of either up-front problem definition nor the pace-limiting effect of observing and iterating on what we have already produced. In fact, we need both of these stages — and both need to be embedded within a shared system. A system that lets us define what we want to learn before we build, so that we set ourselves up to succeed when we are looking for evidence of the ways in which actually worked (and the ways in which didn’t).

Charlie Fountaine published a short list of principles for creating prototypes that actually do something within that socio-technical system, above the banal level of screens with clickable buttons. But that prototyping loop ought to itself be grounded in an evidence-driven semantic environment. Rachel Coldicutt sets out a good overview of such a system here.

Anyone who thinks that AI can replace the need for conceptual clarity at the level of these two documents is trying to sell you something.