Residual Logic: Why the First Move Determines the Game

At an embassy event a journalist chairing a panel offered a familiar line: "AI is like a knife. Only when it is used does it become useful or harmful."

Walk into a hardware store and ask to buy a knife and we are reminded that there is no such thing as 'a knife'. The assistant will ask what you want to do. Butter toast, separate hide from muscle, whittle a stick, open a package?

Every knife is designed for a purpose. And so is AI.

Choices are made for LLMs to encode certain patterns, logic and choices into its output.

And this scaffolding often persists into the final edit.

I'm calling this 'Residual Logic'.

Scaffolding persists from prompt to publication through iterative refinement

It's commonplace to use an LLM to produce a first draft. What follows is a process where the human writer keeps the parts that are useful and makes adjustments and refinements until it fits their requirements.

This editing can produce a usable, even high-quality document in a fraction of the time it took when you had to start with a blank page, but as the document is submitted, acted on or referred to, AI's residual logic has endured. The logic might be fine, but it might hold unseen consequences and influence that were not intended or noticed.

In the transition from blank page to first draft, the LLM creates a logical architecture around the argument, deciding which variables to include and exclude. It determines the sequence of claims and uses these to frame the problem. None of this is offered for argument. It implicitly asks the user to compare the text against infinite alternative framings that were never presented.

It requires the logic of the dog that didn't bark. Sherlock Holmes solved this case by asking why an expected event never happened, while everyone else analysed the evidence that was present. To notice what is missing requires you to hold in mind a model of what should be there, while simultaneously processing what is.

When you edit an LLM draft, the structure arrived before you did. You can question the phrasing, tighten the argument, remove what seems weak. To notice the structures the model did not offer, you would need to imagine the alternative framings it could have provided, the variables it decided were not relevant, the sequences it never considered. You would need to do that while also editing the document you actually have, under time pressure, with other work waiting.

Almost nobody does this and almost nobody can.

So the editing tends to focus on clarity, completeness, and plausibility. If a document already looks coherent, it feels irrational to tear it down and start again. People work with what is in front of them. The result is a document that looks human-owned, reads well, and carries a logic that no one in the organisation fully authored.

"I edited it" is therefore a false assurance, since the human edited a structure that they did not build and may not have examined. The document carries their name while retaining the model's framing. That may still be responsible editing. It is not the same as authorship.

In the age of generative drafting, 'I edited it' may amount to 'I read it and it sounded plausible' when structural review is absent from the editing process.

A writer who begins from a blank page must decide what matters. They must pick a thesis, order it, and live with the implications. With LLMs, an editor who begins from a model's draft is offered a scaffold that already looks reasonable. The work then becomes selection and refinement rather than construction. Over time, people lose practice in structural authorship, because the system is always ready with a plausible structure.

Inherited framing becomes binding without passing through human judgement

The framing used by LLMs reflects averaged patterns from training data; it is not a deliberate judgement about this situation. When a financial model produces a number, the organisation understands that assumptions are embedded. When an LLM produces a document, the assumptions are just as present but arrive disguised as prose.

Framing decisions are a form of policy, enacted through structure rather than stated explicitly.

When a consequential document turns out to have framed a situation wrongly or omitted a material consideration, the question of accountability becomes complicated. The person who edited the document did not make those choices. The system that made those choices cannot be held accountable. The organisation approved an output whose structural logic was never actually owned by anyone.

When structure precedes strategy

The concern is immediate for documents that matter: strategic recommendations, risk assessments, policy positions, board materials, regulatory filings. These are documents where framing determines outcomes and where exclusions can be as consequential as inclusions.

The structure, selected by the LLM, decides what will be talked about. It determines which problems receive executive attention and which remain peripheral. If left unexamined within the review process it will standardise how the organisation thinks about its own reality; structure shapes perception.

A model's default framing could become an organisational default simply because it is a path of low resistance.

Effective governance in this context may require extending the review aperture backward to include the framing and variable selection decisions that precede the visible draft. Right back to the framing the model selected and the choice, conscious or not, to accept that framing without asking what alternatives were never offered.

We may refer to LLM technologies as Artificial Intelligence, but LLMs are not intelligent in any institutional sense since the systems cannot own intent, duty, or accountability. But LLMs are not software either. Their outputs are not stable, repeatable results produced by an inspectable mechanism.

Organisational controls were built for two worlds: deterministic systems and accountable humans. AI belongs to neither, and the control language for that gap does not yet exist.