Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Irving John Good
Later on Good changed his mind:
The paper] “Speculations Concerning the First Ultra-intelligent Machine” (1965) . . . began: “The survival of man depends on the early construction of an ultra-intelligent machine.” Those were his [Good’s] words during the Cold War, and he now suspects that “survival” should be replaced by “extinction.” He thinks that, because of international competition, we cannot prevent the machines from taking over. He thinks we are lemmings. He said also that “probably Man will construct the deus ex machina in his own image.”
→
e.g. insurance companies -- who pays if a driverless car gets in an accident? machine learning algorithms
Drive metros, beat world champions in chess, perform surgery, Suggest ads, deduce we live in Gbg and work at Chalmers, detect credit card fraud Translate websites, recognise speech, make jokes "What do you get when you cross an optic with a mental object? An eye-dea"
Hence, an AI is a machine that is capable of matching or exceeding human performance in most areas, whatever its metaphysical status. So a true AI would be able to converse with us about the sex lives of Hollywood stars, compose passable poetry or prose, design an improved doorknob, guilt trip its friends into coming to visit it more often, create popular cat videos for YouTube, do original AI research, and so on.
And yet there have been many domains that seemed to require common sense that have been taken over by computer programs that demonstrate no such ability: playing chess, answering tricky Jeopardy! questions, translating from one language to another, etc. In the past, it seemed impossible that such feats could be accomplished without showing “true understanding,” and yet algorithms have emerged which succeed at these tasks, all without any glimmer of human-like thought processes.
http://www.defmin.fi/files/3138/MATINE_public_Laakasuo.pdf
A being has a moral status if it is sentient and sapient
Substrate and ontogeny non-discrimination
Intelligence explosion; self-modifying AI won't stop when it has reached human-level intelligence.
Intelligence and meaningful values (to humans) are orthogonal
A superintelligence whose super-goal is to calculate the decimal expansion of pi will never reason itself into benevolence. It would be quite happy to convert all the free matter and energy in the universe (including humans and our habitat) into specialized computers capable only of calculating the digits of pi.
Being highly intelligent doesn't mean that AI would find goals that are meaningful to humans
“Chimpanzees are intelligent, but it's still humans whose actions decide what will happen to them”
Superintelligent AI in power doesn't just let things happen
→ There is no difference for an AI between a sin of commission (doing something bad) and a sin of omission (not doing something good).
But it gets worse: it seems we need to solve nearly all of moral philosophy in order to program a safe AI.
The AIs would be pulling on so many levers of influence all the time that there would be no such thing as a neutral act for them. If they buy a share of stock, they end up helping or hindering sex trafficking in Europe—and they can calculate this effect. In the same way, there is no difference for an AI between a sin of commission (doing something bad) and a sin of omission (not doing something good). For example, imagine someone is getting mugged and murdered on a dark street corner. Why is the mugger there? Because their usual “turf” has been planted with streetlights, at the AI’s instigation. If the streetlights hadn’t been put up, the murder wouldn’t have happened—or maybe a different one would have happened instead. After a very short time in operation, the AI bears personal responsibility for most bad things that happen in the world. Hence, if someone finds themselves in a deadly situation, it will be because of a decision the AI made at some point. For such an active AI, there is no such thing as “letting events just happen.” So we don’t need the AI to be as moral as a human; we need it to be much, much more moral than us, since it’s being put in such an unprecedented position of power.
Try to limit AI's capability ("AI box")
Try to make it friendly from start (Value selection)
For instance, one popular suggestion that reappears periodically is to confine the AI to only answering questions—no manipulators, no robot arms or legs. This suggestion has some merit, but often those who trot it out are trapped in the “Terminator” mode of thinking—if the AI doesn’t have a robot body bristling with guns, then it can’t harm us. This completely fails to protect against socially manipulative AIs, against patient AIs with long time horizons, or against AIs that simply become so essential to human societies and economies that we dare not turn them off.
Another common idea is to have the AI designed as a mere instrument, with no volition of its own, simply providing options to its human controller. But that image of a safe, inert instrument doesn’t scale well: as we’ve seen, humans will be compelled by our slow thinking to put more and more trust in the AI’s decisions. So as the AI’s power grows, we will still need to code safety precautions.
Other approaches, slightly more sophisticated, acknowledge the complexity of human values and attempt to instil them into the AI indirectly. The key features of these designs are social interactions and feedback with humans. Through conversations, the AIs develop their initial morality and eventually converge on something filled with happiness and light and ponies. These approaches should not be dismissed out of hand, but the proposers typically underestimate the difficulty of the problem and project too many human characteristics onto the AI. This kind of intense feedback is likely to produce moral humans.
| Table of Contents | t |
|---|---|
| Exposé | ESC |
| Full screen slides | e |
| Presenter View | p |
| Source Files | s |
| Slide Numbers | n |
| Toggle screen blanking | b |
| Show/hide slide context | c |
| Notes | 2 |
| Help | h |