← Writing

The Awareness Trap

14 April 2026

The bias the regulation cannot fix

The Awareness Trap

AI-generated summary

Foster-Fletcher reads the EU AI Act's Article 14, which requires people overseeing high-risk AI to remain aware of their tendency to over-trust its output. He separates the five distinct cognitive capabilities the clause demands and notes that each degrades independently under routine deployment. Drawing on Laux and Ruschemeier's review of the psychological evidence, he concludes the drafters knew the science and wrote the law anyway: the correction machinery may protect the population over time but not the individual at the moment of decision.

The EU AI Act requires people overseeing high-risk AI to "remain aware" of their tendency to over-trust the output. Article 14 separates five distinct cognitive capabilities across its own clauses: understanding limitations, sustaining vigilance, detecting anomalies, interpreting outputs correctly, and overriding the system under pressure. Each of these degrades independently under the conditions that routine AI deployment creates.

I spent some time with the regulation, with Laux and Ruschemeier's examination of the psychological evidence in the European Journal of Risk Regulation, and with the strongest possible defence of the Act's design. What I found was more interesting, to me, than a regulatory failure.

The drafters knew the science and they wrote the law anyway.

The correction machinery, post-market monitoring, incident reporting, iterative improvement, may protect the population over time. It does not protect the individual at the moment of decision. Article 14 is a design performance standard for providers. It is also an aspirational human performance standard that the science says cannot be reliably met.

The full essay is published on the MKAI archive: https://mkai.org/examinations/automation-bias-eu-ai-act-article-14/

ShareLinkedInXEmail

The analysis

Areas
Responsibility & Liability, Executive Judgement & Access
Themes
Regulation & standards, Governance & oversight
Core question
Can the human oversight Article 14 demands, staying aware of automation bias and overriding the system, actually be delivered by the people the law assigns it to?
Central claim
Article 14 is both a design standard for providers and an aspirational human performance standard that the science says cannot be reliably met, and the drafters wrote it knowing this.
Left open
Whether population-level correction machinery is an acceptable substitute for protecting the individual at the moment of the decision remains open.
Evidence
policy or regulatory documents, conceptual argument grounded in documented cases
Sources
EU AI Act, Article 14, Laux and Ruschemeier, European Journal of Risk Regulation
Entities
EU AI Act (Article 14) (standard), European Union (public body), Laux and Ruschemeier (individual), MKAI (research org)
Concepts introduced
awareness trap
Article form
document analysis, research commentary
Detailed tags
automation bias · human oversight limits · eu ai act article 14 · aspirational performance standards · population versus individual protection
Reader questions
  • What does Article 14 of the EU AI Act actually require of human overseers?
  • Which five cognitive capabilities does Article 14 separate across its clauses?
  • Can people reliably stay aware of their own tendency to over-trust AI output?
  • Did the EU AI Act's drafters know the psychological evidence when they wrote the oversight requirement?
  • Does post-market monitoring protect an individual at the moment of a decision?
  • Why might a thorough regulation still fail to fix automation bias?