The Undesigned Company

Companies have always been authored things.

Badly authored, at times, shaped by politics, inertia, and compromise, but still, at bottom, products of minds.

With LLMs, the modern firm may be the first organisation in history whose internal reasoning includes a significant source of variation that does not originate from minds.

LLM output is pervasive. It sticks because it saves time, the logic usually looks good enough, it's usually ready to use in parts, and it's delivered with confidence. The volume of usage in an organisation means that much of AI's logic creeps through without challenge, and any serious attempt to check and validate each output would clearly undermine the business case of using LLMs.

With that dynamic in place, the modern organisation's development doesn't just start to look undesigned. It starts to look exactly like another model. Nature's.

Natural selection produces organisms that look designed without any designer. Variation enters blindly, what survives is retained, and the reason why is not stored anywhere in the organism. I would argue that the modern LLM-powered organisation is behaving exactly the same way.

LLM Generated content floods the organisation, and the company’s internal archive becomes nothing more than a record of what survived the 'delete' key. It mirrors a biological genome: a DNA sequence carries the adaptation (the 'what'), but it doesn't carry the history of the environmental pressures that forced it to evolve (the 'why').

In this new model, the company adopts reasoning without actually knowing why that logic was chosen over any other. We cannot simply 'reverse-engineer' the thought process, either. As Cynthia Rudin argued in 2019, to truly explain a model's output, you would essentially have to re-run the entire computation. The 'explanation' is just the machine working again, leaving the human observer no closer to the original intent.

In principle, the reviewer could document why each output was retained. At the volumes organisations are operating at, that documentation does not happen.

If the reasoning cannot be recovered, the company is left asking where it came from. This leaves the board with two uncomfortable possibilities. Either the company’s reasoning is increasingly shaped by randomness, or it is the product of a model trained on everyone’s data. The latter produces reasoning that is shared in structure and in tendency with every competitor in the market.

Both paths lead to the same place. A force is shaping the modern enterprise that owes nothing to the minds inside it.

The difference comes down to this: companies that know what they think versus companies that think what survived. The former still author their positions, even if they do so slowly or messily. The latter accumulate theirs like a reef accumulates coral. They grow through retention rather than intent.

One path preserves institutional memory as something recoverable. The other treats it as something emergent. It is a pattern that exists but cannot be traced back to its source. The distinction sounds subtle, but the implications are not. The second company may still look disciplined and high-performing for quite a long time. That is exactly why it is a problem.

Note: Some will object that LLMs are not random. To which I would say that a slot machine is not random either. It runs on a precise algorithm, yet that does not help you to predict what comes out.