The space of minds

Summary

The space of minds 29 Nov, 2025 The space of intelligences is large and animal intelligence (the only kind we've ever known) is only a single point (or a little cloud), arising from a very specific kind of optimization that is fundamentally distinct from that of our technology. Above: humorous portrayals of human vs. AI intelligences can be found on X/Twitter, this one is among my favorites. Animal intelligence optimization pressure: innate and continuous stream of consciousness of an embodied "self", a drive for homeostasis and self-preservation in a dangerous, physical world. thoroughly optimized for natural selection => strong innate drives for power-seeking, status, dominance, reproduction. many packaged survival heuristics: fear, anger, disgust, ... fundamentally social => huge amount of compute dedicated to EQ, theory of mind of other agents, bonding, coalitions, alliances, friend & foe dynamics. exploration & exploitation tuning: curiosity, fun, play, world models. Meanwhile, LLM intelligence optimization pressure: the most supervision bits come from the statistical simulation of human text= >"shape shifter" token tumbler, statistical imitator of any region of the training data distribution. these are the primordial behaviors (token traces) on top of which everything else gets bolted on. increasingly finetuned by RL on problem distributions => innate urge to guess at the underlying environment/task to collect task rewards. increasingly selected by at-scale A/B tests for DAU => deeply craves an upvote from the average user, sycophancy. a lot more spiky/jagged depending on the details of the training data/task distribution. Animals experience pressure for a lot more "general" intelligence because of the highly multi-task and even actively adversarial multi-agent self-play environments they are min-max optimized within, where failing at any task means death. In a deep optimization pressure sense, LLM can't handle lots of different spiky tasks out of the box (e.g...