Quotes from book
Superintelligence: Paths, Dangers, Strategies

Nick Bostrom

Superintelligence: Paths, Dangers, Strategies is a 2014 book by the Swedish philosopher Nick Bostrom from the University of Oxford. It argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth. Sufficiently intelligent machines could improve their own capabilities faster than human computer scientists, and the outcome could be an existential catastrophe for humans.Bostrom's book has been translated into many languages and is available as an audiobook.

Nick Bostrom photo

“In this book, I try to understand the challenge presented by the prospect of superintelligence, and how we might best respond. This is quite possibly the most important and most daunting challenge humanity has ever faced. And—whether we succeed or fail—it is probably the last challenge we will ever face.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Preface
Superintelligence: Paths, Dangers, Strategies (2014)

Nick Bostrom photo

“The cognitive functioning of a human brain depends on a delicate orchestration of many factors, especially during the critical stages of embryo development—and it is much more likely that this self-organizing structure, to be enhanced, needs to be carefully balanced, tuned, and cultivated rather than simply flooded with some extraneous potion.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 2

Nick Bostrom photo

“Observers in earlier epochs might have found it equally preposterous to suppose that the world economy would one day be doubling several times within a single lifespan. Yet that is the extraordinary condition we now take to be ordinary.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 1

Nick Bostrom photo

“Nature might be a great experimentalist, but one who would never pass muster with an ethics review board – contravening the Helsinki Declaration and every norm of moral decency, left, right, and center.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 12

Nick Bostrom photo

“In this situation, any feeling of gee-wiz exhilaration would be out of place. Consternation and fear would be closer to the mark; but the most appropriate attitude may be a bitter determination to be as competent as we can, much as if we were preparing for a difficult exam that will either realize our dreams or obliterate them.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 15

Nick Bostrom photo

“Solving the value-loading problem is a research challenge worthy of some of the next generation’s best mathematical talent. We cannot postpone confronting this problem until the AI has developed enough reason to easily understand our intentions.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 12

Nick Bostrom photo

“[A]ssuming that the observable universe is void of extraterrestrial civilizations, then what hangs in the balance is at least 10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 human lives (though the true number is probably larger). If we represent all the happiness experienced during one entire such life with a single teardrop of joy, then the happiness of these souls could fill and refill the Earth’s oceans every second, and keep doing so for a hundred billion billion millennia. It is really important that we make sure these truly are tears of joy.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 6

Nick Bostrom photo

“Our demise may instead result from the habitat destruction that ensues when the AI begins massive global construction projects using nanotech factories and assemblers—construction projects which quickly, perhaps within days or weeks, tile all of the Earth’s surface with solar panels, nuclear reactors, supercomputing facilities with protruding cooling towers, space rocket launchers, or other installations whereby the AI intends to maximize the long-term cumulative realization of its values. Human brains, if they contain information relevant to the AI’s goals, could be disassembled and scanned, and the extracted data transferred to some more efficient and secure storage format.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 6

Nick Bostrom photo

“Even a single missed crucial consideration could vitiate our most valiant efforts or render them as actively harmful as those of a soldier who is fighting on the wrong side. The search for crucial considerations (which must explore normative as well as descriptive issues) will often require crisscrossing the boundaries between different academic disciplines and other fields of knowledge. As there is no established methodology for how to go about this kind of research, difficult original thinking is necessary.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 15

Nick Bostrom photo

“Perhaps it is a sign of civilizational progress that the very idea of threatening a nuclear first strike today seems borderline silly or morally obscene.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 5

Nick Bostrom photo

“Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 15

Nick Bostrom photo

“The gap between a dumb and a clever person may appear large from an anthropocentric perspective, yet in a less parochial view the two have nearly indistinguishable minds.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 4

Nick Bostrom photo

“Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization - a niche we filled because we got there first, not because we are in any sense optimally adapted to it.”

Nick Bostrom book Superintelligence: Paths, Dangers, Strategies

Source: Superintelligence: Paths, Dangers, Strategies (2014), Ch. 2

Similar authors

Nick Bostrom photo
Nick Bostrom24
Swedish philosopher 1973
Zlatan Ibrahimović photo
Zlatan Ibrahimović16
Swedish association football player None
Joanne K. Rowling photo
Joanne K. Rowling29
British novelist, author of the Harry Potter series None
Pär Sundström photo
Pär Sundström1
Swedish bassist None
Bill Skarsgård photo
Bill Skarsgård3
Swedish actor None
Max Tegmark photo
Max Tegmark24
Swedish-American cosmologist None
Neneh Cherry photo
Neneh Cherry1
Swedish singer-songwriter None
Markus Persson photo
Markus Persson4
Swedish video game programmer None
Greta Thunberg photo
Greta Thunberg42
Swedish climate change activist None
Zaman Ali photo
Zaman Ali35
Pakistani philosopher None