Steven Pinker on AI, Consciousness, and the Rationality of Human Progress

Steven Pinker defines the meaning of life not just as attaining knowledge, but more broadly as achieving fulfillment, encompassing health, stimulation, and access to cultural and social richness. He posits that while our genes aim for propagation, the human brain sets its own goals. Pinker emphasizes that the human capacity for seeking knowledge and applying reason is a fundamental aspect of Homo sapiens, enabling our species to adapt and thrive across the planet. He argues that the refinement of reason in pursuit of human well-being is our primary challenge today, using intellect and knowledge to make discoveries and agreements that benefit all.\n\nThe discussion pivots to artificial intelligence, where Pinker distinguishes between human biological neural networks and current AI deep learning systems. He notes the mysterious nature of human consciousness – the subjective, first-person experience – which remains an open philosophical question for AI. Pinker suggests that current deep learning systems lack a semantic level of understanding, operating more on statistical regularities than true comprehension. He expresses skepticism that sheer network size alone will lead to consciousness or higher semantic reasoning without suitable engineering, though he believes it's possible in principle for silicon-based systems to emulate human intelligence, if desired and economically viable.\n\nPinker adopts a refreshingly rational and positive stance on AI safety, directly challenging the existential threat concerns voiced by figures like Elon Musk and Sam Harris. He dismisses the "AI takeover" scenario, arguing that intelligence does not inherently equate to a "will to power"; an AI's goals are those we program. He also finds the "collateral damage" or "value alignment problem" (e.g., the paperclip maximizer) to be "utterly fanciful," based on the illogical assumption that engineers would create brilliant systems without specifying safety parameters or understanding human intent. Pinker highlights that the core purpose of AI is not destruction, unlike nuclear weapons, making analogies misleading.\n\nHe stresses the importance of the engineering culture, which is inherently safety-oriented, citing the drastic decline in accidental deaths due to engineering advances. Pinker advocates for prioritizing genuine, high-probability threats like climate change, pandemics, and nuclear war over "infinitesimal probabilities" of AI existential risk, which he views as "magical thinking" not grounded in how AI actually works. He warns against the psychological trap of "intellectual thrill" in imagining novel threats, which can lead to paralyzing fear and misallocation of resources, rather than productive action calibrated to actual risks. Pinker concludes by emphasizing the role of reason and the established culture of safety in engineering as crucial safeguards for technological progress.

I'd say aid comes closest but I would amend that to attaining not only knowledge but fulfillment more generally that is life health stimulation access to the living cultural and social world.

We are unusual among animals in the degree to which we acquire knowledge and use it to survive.

There is something a little bit mysterious about the human neural networks which is that each one of us who is a neural network knows that we ourselves are conscious conscious not of a sense of registering our surroundings or even registering our internal state but in having subjective first-person present-tense experience.

Most of the systems don't have a semantic level a level of actual understanding of who did what to who why where how things work what causes what else.

There's no reason to think that sheer problem-solving capability will set that as one of its goals its goals will be whatever we set it its goals as and as long as someone isn't building a megalomaniacal artificial intelligence and there's no reason to think that it would naturally evolve in that direction.

I think analogies between nuclear weapons and artificial intelligence are fundamentally misguided because the whole point of nuclear weapons is to destroy things the point of artificial intelligence is not to destroy things so the analogy is is misleading.

I think these are utterly fanciful in fact I think they're actually self-defeating they first of all assume that we're going to be so brilliant that we can design an artificial intelligence that can cure cancer but so stupid that we don't specify what we mean by curing cancer in enough detail that it won't kill us in the process.

If it's genuinely intelligent it's not going to pursue some goal single-mindedly omitting every other consideration and collateral effect that's not artificial in general intelligence that's that's artificial stupidity.

There's been enormous attention to what are we going to do with the people whose jobs are made obsolete by artificial intelligence but very little attention given to the fact that the jobs that hooni made obsolete are horrible jobs.

A basic tenet of cognitive psychology is that perception of risk and hence perception of fear is driven by imagined ability not by data.