Which humans shall we select to be good role models for AI?

Patrick Sawer writes at The Telegraph that

Dr Stuart Armstrong, of the Future of Humanity Institute at Oxford University, has predicted a future where machines run by artificial intelligence become so indispensable in human lives they eventually make us redundant and take over. "We are almost at the point of generating an AI that is as intelligent as humans."

Indeed, Dr Armstrong warns that the seemingly benign instruction to an AGI to "prevent human suffering", could logically be interpreted by a super computer as "kill all humans", thereby ending suffering all together.

Furthermore, an instruction such as "keep humans safe and happy", could be translated by the remorseless digital logic of a machine as "entomb everyone in concrete coffins on heroin drips".

While that may sound far fetched, Dr Armstrong says the risk is not so low that it can be ignored.

"There is a risk of this kind of pernicious behaviour by a AI," he said, pointing out that the nuances of human language make it all too easily liable to misinterpretation by a computer. "You can give AI controls, and it will be under the controls it was given. But these may not be the controls that were meant."

Dr Armstrong, who was speaking at a debate on artificial intelligence organised in London by the technology research firm Gartner, warns that it will be difficult to tell whether a machine is developing in a benign or deadly direction.

He says an AI would always appear to act in a way that was beneficial to humanity, making itself useful and indispensable - much like the iPhone's Siri, which answers questions and performs simple organisational tasks - until the moment it could logically take over all functions.

"Plans for safe AI must be developed before the first dangerous AI is created," he writes in his book Smarter Than Us: The Rise of Machine Intelligence. "The software industry is worth many billions of dollars, and much effort is being devoted to new AI technologies. "Plans to slow down this rate of development seem unrealistic. So we have to race toward the distant destination of safe AI and get there fast, outrunning the progress of the computer industry."

One solution to the dangers of untrammelled AI suggested by industry experts and researchers is to teach super computers a moral code.

Unfortunately, Dr Armstrong points out, mankind has spent thousands of years debating morality and ethical behaviour without coming up with a simple set of instructions applicable in all circumstances which it can follow.

Imagine then, the difficulty in teaching a machine to make subtle distinctions between right and wrong.

"Humans are very hard to learn moral behaviour from," he says. "They would make very bad role models for AIs."

Read more here.