Researchers at the HJ Institute of Technology and Management (HJITM) are conducting a joint study with KISLAB, the research group led by Professor Hyunchul Ahn at Kookmin University’s Graduate School of Business IT, on how users come to distrust generative AI. The study examines how negative emotional reactions form when generative AI makes mistakes, and the two teams are extending the model into a paper for an international journal.

Because generative AI such as ChatGPT answers in fluent, confident prose, users easily mistake fluency for accuracy. Yet AI cannot structurally avoid ‘hallucinations’ — presenting non-existent information as fact. Where earlier research mostly asked why people keep using AI, this study asks what happens in users’ minds when AI gets it wrong.

To explain this, the researchers propose two new concepts: expectation rigidity, the tendency to keep believing that ‘AI should obviously be accurate’ even after it errs, and cognitive shock, the disruption felt when that expectation breaks. In a scenario experiment with 201 adult generative AI users, each shown either a formatting error or a content-and-source error, people who relied more on AI held more rigid expectations, and rigid expectations led to greater shock when an error appeared. That shock spread into anger more strongly than into dissatisfaction, and more severe errors increased both.

The researchers suggest that AI providers design ‘beneficial friction’ that gently lowers users’ expectations — for example, signals that an answer needs checking — and use different recovery strategies for angry and for disappointed users. HJITM and Professor Ahn’s team previously co-published a study on success factors of corporate AI projects in the Journal of Computer Information Systems (SCIE).

A footnote — the AI that drafted this article also got one character of a researcher’s name wrong at first. Fortunately the editor’s expectations were flexible enough that it ended in a quiet correction rather than anger.