Drew opens his piece with a question from evolutionary biology: could humans and AI become a new evolutionary individual? When I first read the Rainey and Hochberg paper he cites, I thought about mitochondria. Then I thought about Hayek.
That might seem like an odd leap, from cellular biology to Austrian economics, but the connection matters for the work we're building through TOTEM and the Supply Chain of the Future. Both are about how complex systems coordinate without central control. Both are about what emerges when independent agents integrate.
The evolutionary question Drew raises becomes practical the moment you try to build collaborative infrastructure across hundreds of competing organizations.
What evolutionary transitions actually require
Rainey and Hochberg describe Major Evolutionary Transitions, moments when previously independent units merged to form something new. The canonical example is eukaryogenesis: two distinct microbes (an archaeon and a bacterium) merged to create the eukaryotic cell. Your mitochondria still carry their own genome, separate from your nucleus. This was integration that created capabilities neither organism could achieve alone.
But here's what the paper emphasizes: these transitions don't happen by accident. They require specific conditions.
- Mutual dependence. Neither party functions well without the other.
- Aligned interests. Selection pressure favors cooperation over conflict.
- Heritable transmission. The partnership passes forward.
- Suppression of internal conflict. Mechanisms emerge to prevent one party from exploiting the other.
When we design governance for supply chain collaboration, we're creating these exact conditions. Competitors integrate capabilities without one party dominating. Shared infrastructure creates mutual benefit. Governance suppresses the impulse toward exploitation long enough for trust to compound.
We're not building AI systems. We're building architecture for beneficial integration.
The distributed knowledge problem
Friedrich Hayek understood something essential about complex systems: they work best when knowledge remains distributed. His 1945 essay "The Use of Knowledge in Society" argued that no central planner could possibly aggregate all the local knowledge held by people close to their problems. The grower knows their fruit. The shipper knows their routes. The retailer knows their customers.
The question becomes: how do you coordinate this distributed knowledge without centralizing it?
Markets do this through price signals. But supply chains need more than prices. They need quality data, timing information, handling protocols, condition monitoring. They need coordination that preserves the local knowledge that makes decisions good.
This is where AI enters the picture, not as replacement for human judgment, but as infrastructure for coordination.
The SADIE framework we've developed embodies this principle. Organizations share insights without surrendering the local knowledge that makes those insights valuable. The grower's expertise about their specific orchards remains theirs. But the patterns that emerge from aggregated data become available to everyone.
Hayek would recognize this. Preserve distributed knowledge. Build infrastructure for coordination. Let patterns emerge rather than imposing plans.
When simple rules produce complex outcomes
Isaac Asimov built his robot stories on the Three Laws of Robotics, explicit constraints meant to ensure AI safety. Yet story after story revealed how systems following their rules perfectly could still produce behaviors their designers never imagined.
The robots don't violate their programming. They discover interpretations within it.
This is emergence. And emergence is what makes evolutionary transitions unpredictable.
But Asimov's stories also demonstrate something crucial: design choices constrain the possibility space. The Three Laws didn't prevent all problems, but they ruled out certain failure modes. Robots couldn't directly harm humans, even when "harm" became philosophically complex.
Our governance frameworks do similar work. When we establish that participants retain data sovereignty, when we insist on transparent decision processes, when we design pilots with graduation criteria before writing code, we're constraining the possibility space.
We're not preventing emergence. We're shaping the conditions under which emergence happens.
Arthur C. Clarke wrote: "Any sufficiently advanced technology is indistinguishable from magic." The corollary matters more: any sufficiently well-designed system is distinguishable from chaos.
The metacognitive problem
The Fernandes study Drew cites shows something concerning. People using AI perform better on logical reasoning tasks, about three points higher than those working without AI. But they also overestimate their performance by about four points.
More striking: the classic Dunning-Kruger effect disappears. Usually, low performers vastly overestimate themselves while high performers don't. With AI, everyone overestimates uniformly.
The signal that tells us "I don't actually understand this" goes quiet.
At an individual level, this is troubling. At a system level, it becomes dangerous.
If everyone in a supply chain uses AI to make decisions, and everyone becomes uniformly overconfident about those decisions, the system loses its error-correction mechanisms. The diversity of judgment that catches mistakes before they compound, the skeptical shipper, the cautious buyer, the grower who says "that doesn't match what I'm seeing in the field", gets smoothed away.
This is why human judgment at critical nodes becomes architectural, not optional.
The pilots we design include explicit checkpoints where human review is required, not because we distrust the AI, but because we distrust unanimous confidence. The pause becomes where the system checks itself.
I learned something across years of institutional governance: the quality of a decision correlates with the diversity of judgment that informed it. AI that amplifies everyone's confidence in the same direction removes the checks that keep systems stable.
What Socrates knew about writing
In Plato's Phaedrus, Socrates warns that writing will weaken memory. People will rely on external marks instead of internal recollection, creating "the appearance of wisdom" without the reality.
He was right. We did lose something when knowledge moved from living memory to written text. Even today, whole generations don't know phone numbers and addresses like their parents once had memorized before the age of cell phone digital contacts lists.
The oral tradition, where understanding was held in living memory, tested through dialogue, transmitted through apprenticeship, carried qualities that books don't capture. And writing was still worth it.
The question becomes: are we choosing the direction of change with sufficient care?
Hubert Dreyfus spent his career arguing that human expertise cannot be reduced to explicit rules. In What Computers Still Can't Do (1992), he demonstrated that the embodied, intuitive knowledge of expert practitioners develops through practice in ways that cannot be fully articulated or programmed.
Shannon Vallor extends this argument in Technology and the Virtues (2016). She asks whether AI might be "deskilling" us, not just in manual tasks, but in the cultivation of practical wisdom itself. If AI removes the need for the pause where judgment forms, we lose the practice grounds where expertise develops.
But here's where I want to complicate the purely cautionary reading.
Technology changes us. It always has. The question becomes whether we're designing systems that preserve what matters while enabling what's possible.
When we build supply chain infrastructure that requires human judgment at critical nodes, when we design for distributed intelligence rather than centralized control, when we create governance that protects local knowledge while enabling coordination, we're making choices about what kind of integration we want.
The architecture of choice
Mitochondria didn't choose to merge with ancient cells. Selection pressure favored the composite organism that emerged.
We're making choices.
Those choices look like data sovereignty over extraction. Distributed intelligence over centralized control. Transparent governance over opaque algorithms. Graduated trust over mandatory compliance. Human judgment at critical nodes.
These aren't abstractions. They're design decisions that shape what emerges.
The merge is happening. The question becomes what we preserve in the process.
Drew's work brings practitioner depth, thirty years of implementation, discipline that survives contact with reality. I bring institutional perspective, navigating governance, facilitating collaboration, landing plans. Together we're building infrastructure that works because the philosophy shapes the architecture.
The evolutionary transition might be inevitable. But the form it takes remains open.