Trusting your own judgement on 'AI' is a risk

Summary

Trusting your own judgement on 'AI' is a huge risk 9 June 2025 – Baldur Bjarnason (This is loosely based on a couple of social media threads I posted last week, made longer and more tedious with added detail.) One of the major turning points in my life was reading my dad’s copy of Robert Cialdini’s Influence: The Psychology of Persuasion as a teenager. Other highlights of my dad’s library – he was a organisational psychologist before he retired – included books by Fanon, Illich, and Goffman and a bunch on systems thinking and systems theory so, in hindsight, I was probably never not going to be idiosyncratic. But Cialdini’s book was a turning point because it highlighted the very real limitations to human reasoning. No matter how smart you were, the mechanisms of your thinkings could easily be tricked in ways that completely bypassed your logical thinking and could insert ideas and trigger decisions that were not in your best interest. He documented tactics and processes used by salespeople, thieves, and con artists and showed how they could manipulate and trick even smart people. Worse yet, reading through another set of books in my dad’s library – those written by Oliver Sacks – indicated that the complex systems of the brain, the ones that lend themselves to manipulation and disorder, are a big part of what makes us human. But to a self-important asshole teenager, one with an inflated sense of his own intelligence, Cialdini’s book was a timely deflation as it was specifically written as a warning to people to be careful about manipulation. He described it as a process where the mechanics of cognitive optimisation in the brain could be deceptively triggered into a semi-automatic sequence that bypassed your regular judgement – his very eighties metaphor was like that of a tape in your mind that went “click whirr” and played in response to specific stimuli. These are what I tend to call psychological or cognitive hazards. Like the golfer’s sand trap, the only way to...