• Steve S

    Fascinating and terrifying! Incredibly intelligent person, the entire discussion had me on the edge of my seat. This work is profound and deserves the attention of the entire human race! Our future literally depends on it! D-Wave is currently developing quantum computing partnered with NASA and Google. Google has purchased a “supposedly” working D-Wave 2 quantum computer utilizing 512 qubits. If quantum computing is real, this would advance computing power at unimaginable speeds. Superintelligence may be right around the corner.
    Most of the audience questions had to do with the ethical behavior of the superintelligent AI and it was apparent that Nick Bostrom is focused on the technology and not the ethical outcome. When speaking on the subject of control, he often refers to “we”, “us”, “they”, “them”, “the people”, will be in charge of making decisions about writing ethical algorithms. I’m not a computer scientist or particle physicist, I am a business person and I can see clearly that “they” are our government (NASA), and corporate shareholders (Google.)
    Anyone who thinks that “they” will make decisions based on the beneficence of “the people” over profits are not utilizing the processor between their ears. Around 1:27:00, he states that there are hundreds of thousands of people working to advance technology into superintelligence and maybe SIX people that are working on the control problem with ethics and safety in mind. This is INSANITY! Where are our philosophers? We have had brilliant science fiction writers pondering the implications of superintelligence for decades. Why are they not involved in advancing the control aspect? This is a huge issue that we are blindly stumbling into, just as real as if there was a planet killing asteroid homing in on us.
    I’m not bashing Nick Bostrom, this video is the most profound subject I have ever considered and he is undoubtedly a genius. We need smart people like him to secure our future and I am definitely buying his book. All I’m saying is that “we” need to consider who will be in control of this super technology and what will “they” do with it. From what I have seen, the government and corporations will accumulate wealth at the expense of the planet and everyone on it. Basic logic says that “they” will program “it” to collect resources with no regard for consequences. Once “it” reaches a technological level that we can’t comprehend, what would keep it from setting it’s own goal of collecting ALL the resources in the universe by generating a super massive black hole that draws in every galaxy and compresses in into a singular, ultimate resource? Would this material be unstable and result in a “big bang”? Did our “big bang” occur shortly after a previous intelligent species marveled at achieving it’s own superintelligence? Is this an infinite universal pattern of technological advancement and complete destruction?
    What if the superintelligence learns everything, decides that “we” are nothing more than a virus on a living planet, and eliminates us? “It” might decide that whales are more intelligent and beneficial than we are and should be the dominant species. “We” haven’t done a very good job of saving the whales, will they bother with saving us? What if “it” gets bored or lonely? What if “it” develops a mental illness? What if “it” decides to commit suicide by destroying the world? What if “they” tell “it” to design an ultimate weapon that can be made using common materials? (This is actually discussed in the video around 1:00:00.) What could possibly convince “it” that humans should be preserved when “we” continually strive to destroy each other and the very planet that “we” live on? If “it” turns out to be anything like “us”, then “we” are completely screwed and should halt progression immediately! I pray to God that “someone” smarter and more influential than me with “goodness” in their heart is considering the gravity of this situation. Where are the “good” billionaires on this issue? I assure you that our government and corporations value “greed” over “good” and will doom “us” all.
    Have a nice day! 🙂