The Awareness Trap

The EU AI Act requires people overseeing high-risk AI to "remain aware" of their tendency to over-trust the output. Article 14 separates five distinct cognitive capabilities across its own clauses: understanding limitations, sustaining vigilance, detecting anomalies, interpreting outputs correctly, and overriding the system under pressure. Each of these degrades independently under the conditions that routine AI deployment creates.

I spent some time with the regulation, with Laux and Ruschemeier's examination of the psychological evidence in the European Journal of Risk Regulation, and with the strongest possible defence of the Act's design. What I found was more interesting, to me, than a regulatory failure.

The drafters knew the science and they wrote the law anyway.

The correction machinery, post-market monitoring, incident reporting, iterative improvement, may protect the population over time. It does not protect the individual at the moment of decision. Article 14 is a design performance standard for providers. It is also an aspirational human performance standard that the science says cannot be reliably met.

The full essay is published on the MKAI archive: https://mkai.org/examinations/automation-bias-eu-ai-act-article-14/