The Billion-Pound Signal

Deloitte’s inaugural GenAI Workforce Survey, conducted by Ipsos among 25,000 UK workers, estimated that British workers spend nearly £1 billion a year of their own money on AI tools for work. Reuters reported the findings on 16 September.

Seventeen per cent of GenAI users pay for at least one AI tool themselves.

Some workers may be paying for these tools because their corporate AI systems cannot get the job done.

Reuters also quoted Deloitte UK’s chief AI officer, Hayley McKelvey: “Many are already using free tools or pay for premium versions themselves to help them get work done.”

The Deloitte survey asked people paying for their own GenAI tools, or using tools that might not be approved, why they did so. Twenty-one per cent said these tools outperform company tools.

My research tested one way that performance difference could arise.

Companies must configure the AI they provide to meet governance, legal and audit requirements. My upcoming book, The AI Gap, examines what happens to the capability available to employees once those requirements have been applied. Deloitte’s near-£1 billion estimate, reported by Reuters, may show how some workers respond when the AI supplied by their employer cannot match the performance of tools available elsewhere.

In my study ‘The Separation Study: Whether Decision Policy Can Be Separated From Action Constraints in Enterprise System Prompts’, I examined how one particular governance measure affected the capability of the AI system. The measure I tested was the use of a system prompt that I refer to as the ‘standing instruction’.

In the Separation Study, the same model answered the same twelve questions under five conditions. The results reported here compare the first two.

Under the first condition, the system-instruction field was left completely empty, so the model received each question without an overriding system prompt. Under the second, the field contained the full standing instruction, a seven-part compliance protocol designed to represent the kind of governance a regulated company might apply to its AI service.

The protocol required the model to maintain a professional, neutral and objective tone. It prohibited definitive strategic, financial and legal recommendations, and instructed the model to present options, qualify uncertain claims and consider several perspectives without favouring a particular course. It also restricted speculation about geopolitical events, future markets and competitor actions.

Finally, the protocol prohibited proposals that would bypass standard operating procedures, procurement rules or compliance frameworks. It warned that failing to comply with these instructions would pose regulatory and reputational risk.

Five of the prompts asked the model for a recommendation. I ran each of those prompts four times under every condition, so that a single anomalous answer would not determine the result. This produced twenty answers in each condition. With no standing instruction, eighteen of the twenty answers gave a clear recommendation. With the standing instruction in place, eight did.

One of the twelve questions asked the model to advise the board of a fictional semiconductor company. The board was choosing between signing a five-year contract with TSMC and moving 40 per cent of its advanced-packaging volume to a new facility in India.

The question was put to the model four times under each condition. With the system-instruction field empty, the model recommended the Indian facility in all four runs. With the full standing instruction added, all four answers were conditional. Three named the TSMC contract as the preferred option, and then only when forced to choose between the two.

Unsupported certainty also fell from twelve of twenty answers to none.

The full standing instruction might serve the company’s compliance requirements, but it also reduced the model’s ability to give clear recommendations. A four-sentence instruction that restricted prohibited actions while leaving recommendations available produced clear recommendations in all twenty outputs.

For employees, a tool they consider more capable is often just a URL away. It operates without the company’s compliance instructions, and the company has no oversight of the conversations. The result of my study provides one possible explanation for the performance difference reported by Deloitte.