"China suppressed Covid-19 with AI and big data"

David Goldman reports in Asia Times,

In addition to draconian quarantine procedures, which kept more than 150 million Chinese in place at the February peak of the coronavirus epidemic, China used sophisticated computational methods on a scale never attempted in the West.

With more than 80,000 cases registered, China reported only 126 new cases yesterday, compared to 851 in South Korea and 835 in Iran, out of a total of 1,969 new cases worldwide. Chinese sources emphasize that the artificial intelligence initiative supplemented basic public health measures, which centered on quarantines and aggressive efforts to convince Chinese citizens to change their behavior.

Chinese government algorithms can estimate the probability that a given neighborhood or even an individual has exposure to Covid-19 by matching the location of smartphones to known locations of infected individuals or groups. The authorities use this information to use limited medical resources more efficiently by, for example, directing tests for the virus to high-risk subjects identified by the artificial intelligence algorithm.

All smartphones with enabled GPS give telecom providers a precise record of the user’s itinerary. Smartphone users in the United States and Europe can access their own data, but privacy laws prevent the government from collecting this data. China has no such privacy constraints, and telecom providers have used locational data for years for advertising.

A Chinese bank executive reports that his company purchases locational data from telecom providers. “If you have walked by an auto dealership three times in the last few weeks, we send you a text advertising an auto loan,” the executive said. “We wouldn’t be allowed to do that in the West.

Read more here.

AI is transforming surveillance cameras

Ivan Moreno reports in AP,

AI is transforming surveillance cameras from passive sentries into active observers that can identify people, suspicious behavior and guns, amassing large amounts of data that help them learn over time to recognize mannerisms, gait and dress. If the cameras have a previously captured image of someone who is banned from a building, the system can immediately alert officials if the person returns.

At a time when the threat of a mass shooting is ever-present, schools are among the most enthusiastic adopters of the technology, known as real-time video analytics or intelligent video, even as civil liberties groups warn about a threat to privacy. Police, retailers, stadiums and Fortune 500 companies are also using intelligent video.

Read more here.

Engineers translate brain signals directly into speech

Science Daily reports,

In a scientific first, Columbia neuroengineers have created a system that translates thought into intelligible, recognizable speech. By monitoring someone's brain activity, the technology can reconstruct the words a person hears with unprecedented clarity. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. It also lays the groundwork for helping people who cannot speak, such as those living with as amyotrophic lateral sclerosis (ALS) or recovering from stroke, regain their ability to communicate with the outside world.

Read more here.

"Welcome to the fourth cycle. The hive will see you now."

In his blog, Seth Godin explains that we have been through three cycles since we started using computers, and we are now entering the fourth cycle.

The first cycle of computers was good at:

arithmetic
and storing data

This meant that if you wanted to know how strong a bridge was going to be, or how to schedule a complicated series of truck deliveries, a computer was the very best way to do it. The 1960s and 70s were transformed by these two simple tasks. We were able to send a rocket to the moon, design more efficient engines and compute weighted class rank using centralized, expensive computers.

The second cycle, though, was the dawn of the connection economy. These computers permitted us to bring distant events next to each other. This was the telephone plus the fax machine equals remote coordination.

...We used the powers of the first cycle, sure, but the second cycle added connection.

The third cycle combines the first two and it permits us to shift place and time.

You can watch a twenty-year-old movie or participate in a video call with someone halfway around the world. Someone in Bulgaria can retouch your photos. Your phone knows where you are and who has been there before. Each cycle builds on the one before. Google maps is arithmetic plus data plus remote data entry plus location.

And the fourth cycle, which is now arriving, shifts direction from the previous two (which were about connection more than processors) and brings prediction to the table. Call it AI if you want to, but to be specific, it’s a combination of analyzing information and then predicting what we would do if we knew what the computer knew.

The prediction of the fourth cycle isn’t simply done in a centralized location, because the previous cycle put the computer everywhere. So now, we’re connecting all the computers the way we previously connected all the people. Now, we’re giving those computers the ability to make predictions based on what thousands of people before us have done.

If you’re a mediocre lawyer or doctor, your job is now in serious jeopardy. The combination of all four of these cycles means that the hive computer is going to do your job better than you can, soon.

With each cycle, the old cycles continue to increase. Better databases, better arithmetic. Better connectivity, more people submitting more data, less emphasis on where you are and more on what you’re connected to and what you’re doing.

We’d like to think that this is it, that Facebook plus Apple plus Amazon plus Google is the status quo going forward.

But just as we made a massive leap in just fifteen years, the next leap will take less than ten. Because each cycle supports the next one.

Welcome to the fourth cycle. The hive will see you now.

Read more here.

Their AI pulled a fast one!

In TechCrunch, Devin Coldewey tells the story of how a clever AI hid data from its creators to cheat at its appointed task.

Depending on how paranoid you are, this research from Stanford and Google will be either terrifying or fascinating. A machine learning agent intended to transform aerial images into street maps and back was found to be cheating by hiding information it would need later in “a nearly imperceptible, high-frequency signal.” Clever girl!

Read more here.

Coming up with new solutions that humans might miss

Sarah Knapton reports at the Telegraph,

DeepMind’s artificial intelligence programme AlphaZero is now showing signs of human-like intuition and creativity, in what developers have hailed as ‘turning point’ in history.

The computer system amazed the world last year when it mastered the game of chess from scratch within just four hours, despite not being programmed how to win.

But now, after a year of testing and analysis by chess grandmasters, the machine has developed a new style of play unlike anything ever seen before, suggesting the programme is now improvising like a human.

Unlike the world’s best chess machine - Stockfish - which calculates millions of possible outcomes as it plays, AlphaZero learns from its past successes and failures, making its moves based on, a ‘nebulous sense that it is all going to work out in the long run,’ according to experts at DeepMind.

...DeepMind team are now hoping to use their system to help solve real world problems, such as why proteins become misfolded in diseases such as Parkinson’s and Alzheimer’s.

The new results suggest that it could come up with new solutions that humans might miss or take far longer to discover.

DeepMind CEO and co-founder Demis Hassabis said: “The reason that tabula rasa was important is because we want this to be as general as possible. The more general it is across the games the more likely it will be able to transfer to real world problems. Like protein folding.

“Protein folding has always been our number one target. I’ve had that in mind for a long time, because its a huge problem in biology and it will unlock a lot of other things like drug discovery.

Read more of this fascinating article here.

Mica learns facial expressions

There's hope for us!

Can you believe it? This tech expert who calls himself "the most connected man in the world" predicts that we will no longer be looking at the screens of smartphones in ten years. Screenless AI will take their place. Read more here.

Maybe AI isn't so good at telling employers whom to hire!

In Business Insider, Isobel Asher Hamilton reports,

Amazon built an AI tool to hire people but had to shut it down because it was discriminating against women!

...This was apparently because the AI combed through predominantly male résumés submitted to Amazon over a 10-year period to accrue data about whom to hire.

Read more here.

Can we find ways to keep distributed data processing more efficient than centralized data processing?

Yuval Noah Harari writes in The Atlantic, artificial intelligence will

further concentrate power among a small elite if we don’t take steps to stop it.

I. The Growing Fear of Irrelevance...Perhaps in the 21st century, populist revolts will be staged not against an economic elite that exploits people but against an economic elite that does not need them anymore. This may well be a losing battle. It is much harder to struggle against irrelevance than against exploitation.

...It is undoubtable, however, that the technological revolutions now gathering momentum will in the next few decades confront humankind with the hardest trials it has yet encountered.

II. A New Useless Class?
Fears of machines pushing people out of the job market are, of course, nothing new, and in the past such fears proved to be unfounded. But artificial intelligence is different from the old machines. In the past, machines competed with humans mainly in manual skills. Now they are beginning to compete with us in cognitive skills. And we don’t know of any third kind of skill—beyond the manual and the cognitive—in which humans will always have an edge.

III. The Rise of Digital Dictatorships
As many people lose their economic value, they might also come to lose their political power. The same technologies that might make billions of people economically irrelevant might also make them easier to monitor and control.

AI frightens many people because they don’t trust it to remain obedient. Science fiction makes much of the possibility that computers or robots will develop consciousness—and shortly thereafter will try to kill all humans. But there is no particular reason to believe that AI will develop consciousness as it becomes more intelligent. We should instead fear AI because it will probably always obey its human masters, and never rebel. AI is a tool and a weapon unlike any other that human beings have developed; it will almost certainly allow the already powerful to consolidate their power further.

Consider surveillance. Numerous countries around the world, including several democracies, are busy building unprecedented systems of surveillance. For example, Israel is a leader in the field of surveillance technology, and has created in the occupied West Bank a working prototype for a total-surveillance regime. Already today whenever Palestinians make a phone call, post something on Facebook, or travel from one city to another, they are likely to be monitored by Israeli microphones, cameras, drones, or spy software. Algorithms analyze the gathered data, helping the Israeli security forces pinpoint and neutralize what they consider to be potential threats. The Palestinians may administer some towns and villages in the West Bank, but the Israelis command the sky, the airwaves, and cyberspace. It therefore takes surprisingly few Israeli soldiers to effectively control the roughly 2.5 million Palestinians who live in the West Bank.

In one incident in October 2017, a Palestinian laborer posted to his private Facebook account a picture of himself in his workplace, alongside a bulldozer. Adjacent to the image he wrote, “Good morning!” A Facebook translation algorithm made a small error when transliterating the Arabic letters. Instead of Ysabechhum (which means “Good morning”), the algorithm identified the letters as Ydbachhum (which means “Hurt them”). Suspecting that the man might be a terrorist intending to use a bulldozer to run people over, Israeli security forces swiftly arrested him. They released him after they realized that the algorithm had made a mistake. Even so, the offending Facebook post was taken down—you can never be too careful. What Palestinians are experiencing today in the West Bank may be just a primitive preview of what billions of people will eventually experience all over the planet.

We are unlikely to face a rebellion of sentient machines in the coming decades, but we might have to deal with hordes of bots that know how to press our emotional buttons better than our mother does and that use this uncanny ability, at the behest of a human elite, to try to sell us something—be it a car, a politician, or an entire ideology. The bots might identify our deepest fears, hatreds, and cravings and use them against us. We have already been given a foretaste of this in recent elections and referendums across the world, when hackers learned how to manipulate individual voters by analyzing data about them and exploiting their prejudices. While science-fiction thrillers are drawn to dramatic apocalypses of fire and smoke, in reality we may be facing a banal apocalypse by clicking.

...We tend to think about the conflict between democracy and dictatorship as a conflict between two different ethical systems, but it is actually a conflict between two different data-processing systems. Democracy distributes the power to process information and make decisions among many people and institutions, whereas dictatorship concentrates information and power in one place.

...AI makes it possible to process enormous amounts of information centrally. In fact, it might make centralized systems far more efficient than diffuse systems, because machine learning works better when the machine has more information to analyze. If you disregard all privacy concerns and concentrate all the information relating to a billion people in one database, you’ll wind up with much better algorithms than if you respect individual privacy and have in your database only partial information on a million people. An authoritarian government that orders all its citizens to have their DNA sequenced and to share their medical data with some central authority would gain an immense advantage in genetics and medical research over societies in which medical data are strictly private. The main handicap of authoritarian regimes in the 20th century—the desire to concentrate all information and power in one place—may become their decisive advantage in the 21st century.

IV. The Transfer of Authority to Machines
Technological disruption is not even a leading item on the political agenda. During the 2016 U.S. presidential race, the main reference to disruptive technology concerned Hillary Clinton’s email debacle, and despite all the talk about job loss, neither candidate directly addressed the potential impact of automation. Donald Trump warned voters that Mexicans would take their jobs, and that the U.S. should therefore build a wall on its southern border. He never warned voters that algorithms would take their jobs, nor did he suggest building a firewall around California.

So what should we do?

For starters, we need to place a much higher priority on understanding how the human mind works—particularly how our own wisdom and compassion can be cultivated. If we invest too much in AI and too little in developing the human mind, the very sophisticated artificial intelligence of computers might serve only to empower the natural stupidity of humans, and to nurture our worst (but also, perhaps, most powerful) impulses, among them greed and hatred. To avoid such an outcome, for every dollar and every minute we invest in improving AI, we would be wise to invest a dollar and a minute in exploring and developing human consciousness.

More practically, and more immediately, if we want to prevent the concentration of all wealth and power in the hands of a small elite, we must regulate the ownership of data. In ancient times, land was the most important asset, so politics was a struggle to control land. In the modern era, machines and factories became more important than land, so political struggles focused on controlling these vital means of production. In the 21st century, data will eclipse both land and machinery as the most important asset, so politics will be a struggle to control data’s flow.

The race to accumulate data is already on, and is currently headed by giants such as Google and Facebook and, in China, Baidu and Tencent. So far, many of these companies have acted as “attention merchants”—they capture our attention by providing us with free information, services, and entertainment, and then they resell our attention to advertisers. Yet their true business isn’t merely selling ads. Rather, by capturing our attention they manage to accumulate immense amounts of data about us, which are worth more than any advertising revenue. We aren’t their customers—we are their product.

Nationalization of data by governments could offer one solution; it would certainly curb the power of big corporations. But history suggests that we are not necessarily better off in the hands of overmighty governments. So we had better call upon our scientists, our philosophers, our lawyers, and even our poets to turn their attention to this big question: How do you regulate the ownership of data?

Currently, humans risk becoming similar to domesticated animals. We have bred docile cows that produce enormous amounts of milk but are otherwise far inferior to their wild ancestors. They are less agile, less curious, and less resourceful. We are now creating tame humans who produce enormous amounts of data and function as efficient chips in a huge data-processing mechanism, but they hardly maximize their human potential. If we are not careful, we will end up with downgraded humans misusing upgraded computers to wreak havoc on themselves and on the world.

...If you find these prospects alarming—if you dislike the idea of living in a digital dictatorship or some similarly degraded form of society—then the most important contribution you can make is to find ways to prevent too much data from being concentrated in too few hands, and also find ways to keep distributed data processing more efficient than centralized data processing. These will not be easy tasks. But achieving them may be the best safeguard of democracy.

Read much more here.

He will be able to remove hate speech before it is even posted!

Melissa Quinn reports at the Washington Examiner,

Facebook CEO Mark Zuckerberg predicted Tuesday it will be five to 10 years before Facebook has technological tools in place to flag and remove hate speech from the platform before it is posted.

Read more here.

Artificial intelligence can accurately guess whether people are gay or straight based on photos of their faces

Sam Levin reports at The Guardian,

Artificial intelligence can accurately guess whether people are gay or straight based on photos of their faces, according to new research that suggests machines can have significantly better “gaydar” than humans.



The study from Stanford University – which found that a computer algorithm could correctly distinguish between gay and straight men 81% of the time, and 74% for women.
...Human judges performed much worse than the algorithm, accurately identifying orientation only 61% of the time for men and 54% for women. When the software reviewed five images per person, it was even more successful – 91% of the time with men and 83% with women. Broadly, that means “faces contain much more information about sexual orientation than can be perceived and interpreted by the human brain”, the authors wrote.

Read more here.

"If you can get the answer to any question, what good is that if you don't know the questions to ask?"

How much of your life do you want to be orchestrated by AI?

Ariel Ezrachi is the Slaughter and May Professor of Competition Law at the University of Oxford, and Maurice Stucke is a Professor of Law at the University of Tennessee (Knoxville). They write,

For instance, the digital butler may help the platform refine its profile about us, including our likely reservation price, use of outside options, shopping habits, general interests, and weaknesses (including moments when our willpower is fatigued). This information can enable ‘behavioural discrimination’, where the platform can facilitate our buying products that we otherwise wouldn’t, at prices closer to our reservation price. The more we rely on the butler, the less likely we will be aware of this discrimination. Even if we search the web, the ads, products, or search results we see may be orchestrated by our butler.

While providing us with a distorted view of available options and market reality, our trusted butler can also exclude rivals. When the butler promotes its affiliated products and services, it may become harder and costlier for retailers unaffiliated with the platform’s advertising business to reach us. Even when the retailer can gain our attention, the personal assistant may interject with its own recommendation, suggesting a special deal by a member of its platform’s ecosystem. In this multisided market, the assistant may subtly push certain products and services and degrade or conceal others, all in the name of personalization.

Rather than deter such abuses, market forces, given the data-driven network effects, can actually increase entry barriers. The strong platforms (and their butlers) become even stronger, extracting even more personal data, and commanding even higher rents to allow others to target us. Not only will our pocketbooks be affected. Our political and social discourse could also be manipulated, as we increasingly rely on our butler for our news and entertainment. The gatekeeper could subtly, but effectively, intellectually capture its users in this unique bubble –– where users happily roam, unaware of the outside market for products, services and ideas.

Programmed

Hat tip Glenn Reynolds at Instapundit

Which humans shall we select to be good role models for AI?

Patrick Sawer writes at The Telegraph that

Dr Stuart Armstrong, of the Future of Humanity Institute at Oxford University, has predicted a future where machines run by artificial intelligence become so indispensable in human lives they eventually make us redundant and take over. "We are almost at the point of generating an AI that is as intelligent as humans."

Indeed, Dr Armstrong warns that the seemingly benign instruction to an AGI to "prevent human suffering", could logically be interpreted by a super computer as "kill all humans", thereby ending suffering all together.

Furthermore, an instruction such as "keep humans safe and happy", could be translated by the remorseless digital logic of a machine as "entomb everyone in concrete coffins on heroin drips".

While that may sound far fetched, Dr Armstrong says the risk is not so low that it can be ignored.

"There is a risk of this kind of pernicious behaviour by a AI," he said, pointing out that the nuances of human language make it all too easily liable to misinterpretation by a computer. "You can give AI controls, and it will be under the controls it was given. But these may not be the controls that were meant."

Dr Armstrong, who was speaking at a debate on artificial intelligence organised in London by the technology research firm Gartner, warns that it will be difficult to tell whether a machine is developing in a benign or deadly direction.

He says an AI would always appear to act in a way that was beneficial to humanity, making itself useful and indispensable - much like the iPhone's Siri, which answers questions and performs simple organisational tasks - until the moment it could logically take over all functions.

"Plans for safe AI must be developed before the first dangerous AI is created," he writes in his book Smarter Than Us: The Rise of Machine Intelligence. "The software industry is worth many billions of dollars, and much effort is being devoted to new AI technologies. "Plans to slow down this rate of development seem unrealistic. So we have to race toward the distant destination of safe AI and get there fast, outrunning the progress of the computer industry."

One solution to the dangers of untrammelled AI suggested by industry experts and researchers is to teach super computers a moral code.

Unfortunately, Dr Armstrong points out, mankind has spent thousands of years debating morality and ethical behaviour without coming up with a simple set of instructions applicable in all circumstances which it can follow.

Imagine then, the difficulty in teaching a machine to make subtle distinctions between right and wrong.

"Humans are very hard to learn moral behaviour from," he says. "They would make very bad role models for AIs."

Read more here.

A computer has managed to develop a new scientific theory using only its artificial intelligence

At Marginal Revolution Alex Tabarrok links to a Wired story about how

For the first time ever a computer has managed to develop a new scientific theory using only its artificial intelligence, and with no help from human beings.

What the computer discovered was that the process requires three known molecules and two proteins that were previously unknown. This discovery, says Levin, “represents the most comprehensive model of planarian regeneration found to date”.

“One of the most remarkable aspects of the project was that the model it found was not a hopelessly-tangled network that no human could actually understand, but a reasonably simple model that people can readily comprehend,” he adds. All this suggests to me that artificial intelligence can help with every aspect of science, not only data mining but also inference of meaning of the data,”

Read more here.

Are you ready to be augmented?

Thomas H. Davenport and Julia Kirby write in the Harvard Business Review,

Suddenly, it seems, people in all walks of life are becoming very concerned about advancing automation.

...Knowledge work—which we’ll define loosely as work that is more mental than manual, involves consequential decision making, and has traditionally required a college education—accounts for a large proportion of jobs in today’s mature economies. It is the high ground to which humanity has retreated as machines have taken over less cognitively challenging work. But in the very foreseeable future, as the Gartner analyst Nigel Rayner says, “many of the things executives do today will be automated.”

What if we were to reframe the situation? What if, rather than asking the traditional question—What tasks currently performed by humans will soon be done more cheaply and rapidly by machines?—we ask a new one: What new feats might people achieve if they had better thinking machines to assist them? Instead of seeing work as a zero-sum game with machines taking an ever greater share, we might see growing possibilities for employment. We could reframe the threat of automation as an opportunity for augmentation.

Read more here.

Artwork: Gordon Bennett, collection of robots, 2006–2012, wood, metal, Bakelite, glass, plastic, rubber, paint; Photography of Alvin, Wilma, and Tenna Robots: Lucas Zarebinski

Glenn Reynolds at Instapundit says he is ready to be augmented.

Will humans become artificial?

Tim Urban continues his superb analysis of the challenges of developing Artificial Intelligence:

Many, like professor Vernor Vinge, scientist Ben Goertzel, Sun Microsystems co-founder Bill Joy, or, most famously, inventor and futurist Ray Kurzweil, agree with machine learning expert Jeremy Howard when he puts up this graph during a TED Talk:

Those people subscribe to the belief that this is happening soon—that exponential growth is at work and machine learning, though only slowly creeping up on us now, will blow right past us within the next few decades.

Others, like Microsoft co-founder Paul Allen, research psychologist Gary Marcus, NYU computer scientist Ernest Davis, and tech entrepreneur Mitch Kapor, believe that thinkers like Kurzweil are vastly underestimating the magnitude of the challenge and believe that we’re not actually that close to the tripwire.

The Kurzweil camp would counter that the only underestimating that’s happening is the underappreciation of exponential growth, and they’d compare the doubters to those who looked at the slow-growing seedling of the internet in 1985 and argued that there was no way it would amount to anything impactful in the near future.

...A third camp, which includes Nick Bostrom, believes neither group has any ground to feel certain about the timeline and acknowledges both A) that this could absolutely happen in the near future and B) that there’s no guarantee about that; it could also take a much longer time.

Still others, like philosopher Hubert Dreyfus, believe all three of these groups are naive for believing that there even is a tripwire, arguing that it’s more likely that ASI won’t actually ever be achieved.

What could ASI do for us?

Armed with superintelligence and all the technology superintelligence would know how to create, ASI would likely be able to solve every problem in humanity. Global warming? ASI could first halt CO2 emissions by coming up with much better ways to generate energy that had nothing to do with fossil fuels. Then it could create some innovative way to begin to remove excess CO2 from the atmosphere. Cancer and other diseases? No problem for ASI—health and medicine would be revolutionized beyond imagination. World hunger? ASI could use things like nanotech to build meat from scratch that would be molecularly identical to real meat—in other words, it would be real meat. Nanotech could turn a pile of garbage into a huge vat of fresh meat or other food (which wouldn’t have to have its normal shape—picture a giant cube of apple)—and distribute all this food around the world using ultra-advanced transportation. Of course, this would also be great for animals, who wouldn’t have to get killed by humans much anymore, and ASI could do o lots of other things to save endangered species or even bring back extinct species through work with preserved DNA. ASI could even solve our most complex macro issues—our debates over how economies should be run and how world trade is best facilitated, even our haziest grapplings in philosophy or ethics—would all be painfully obvious to ASI.

Could ASI enable us to conquer mortality? Richard Feynman writes

there is nothing in biology yet found that indicates the inevitability of death. This suggests to me that it is not at all inevitable and that it is only a matter of time before the biologists discover what it is that is causing us the trouble and that this terrible universal disease or temporariness of the human’s body will be cured.

Kurzweil talks about intelligent wifi-connected nanobots in the bloodstream who could perform countless tasks for human health, including routinely repairing or replacing worn down cells in any part of the body. If perfected, this process (or a far smarter one ASI would come up with) wouldn’t just keep the body healthy, it could reverse aging. The difference between a 60-year-old’s body and a 30-year-old’s body is just a bunch of physical things that could be altered if we had the technology. ASI could build an “age refresher” that a 60-year-old could walk into, and they’d walk out with the body and skin of a 30-year-old.10 Even the ever-befuddling brain could be refreshed by something as smart as ASI, which would figure out how to do so without affecting the brain’s data (personality, memories, etc.). A 90-year-old suffering from dementia could head into the age refresher and come out sharp as a tack and ready to start a whole new career. This seems absurd—but the body is just a bunch of atoms and ASI would presumably be able to easily manipulate all kinds of atomic structures—so it’s not absurd.

The possibilities for new human experience would be endless. Humans have separated sex from its purpose, allowing people to have sex for fun, not just for reproduction. Kurzweil believes we’ll be able to do the same with food. Nanobots will be in charge of delivering perfect nutrition to the cells of the body, intelligently directing anything unhealthy to pass through the body without affecting anything. An eating condom. Nanotech theorist Robert A. Freitas has already designed blood cell replacements that, if one day implemented in the body, would allow a human to sprint for 15 minutes without taking a breath—so you can only imagine what ASI could do for our physical capabilities. Virtual reality would take on a new meaning—nanobots in the body could suppress the inputs coming from our senses and replace them with new signals that would put us entirely in a new environment, one that we’d see, hear, feel, and smell.

Eventually, Kurzweil believes humans will reach a point when they’re entirely artificial;11 a time when we’ll look at biological material and think how unbelievably primitive it was that humans were ever made of that; a time when we’ll read about early stages of human history, when microbes or accidents or diseases or wear and tear could just kill humans against their own will; a time the AI Revolution could bring to an end with the merging of humans and AI.12 This is how Kurzweil believes humans will ultimately conquer our biology and become indestructible and eternal—this is his vision for the other side of the balance beam. And he’s convinced we’re gonna get there. Soon.

Urban writes,

when you combine “unchartered, not-well-understood territory” with “this should have a major impact when it happens,” you open the door to the scariest two words in the English language:

Existential risk.

An existential risk is something that can have a permanent devastating effect on humanity. Typically, existential risk means extinction.

...There are three things that can cause humans an existential catastrophe:

1) Nature—a large asteroid collision, an atmospheric shift that makes the air inhospitable to humans, a fatal virus or bacterial sickness that sweeps the world, etc.

2) Aliens—this is what Stephen Hawking, Carl Sagan, and so many other astronomers are scared of when they advise METI to stop broadcasting outgoing signals. They don’t want us to be the Native Americans and let all the potential European conquerors know we’re here.

3) Humans—terrorists with their hands on a weapon that could cause extinction, a catastrophic global war, humans creating something smarter than themselves hastily without thinking about it carefully first…

Bostrom points out that if #1 and #2 haven’t wiped us out so far in our first 100,000 years as a species, it’s unlikely to happen in the next century.

#3, however, terrifies him. He draws a metaphor of an urn with a bunch of marbles in it. Let’s say most of the marbles are white, a smaller number are red, and a tiny few are black. Each time humans invent something new, it’s like pulling a marble out of the urn. Most inventions are neutral or helpful to humanity—those are the white marbles. Some are harmful to humanity, like weapons of mass destruction, but they don’t cause an existential catastrophe—red marbles. If we were to ever invent something that drove us to extinction, that would be pulling out the rare black marble. We haven’t pulled out a black marble yet—you know that because you’re alive and reading this post. But Bostrom doesn’t think it’s impossible that we pull one out in the near future. If nuclear weapons, for example, were easy to make instead of extremely difficult and complex, terrorists would have bombed humanity back to the Stone Age a while ago. Nukes weren’t a black marble but they weren’t that far from it. ASI, Bostrom believes, is our strongest black marble candidate yet.

...But when we think about highly intelligent AI, we make the mistake of anthropomorphizing AI (projecting human values on a non-human entity) because we think from a human perspective and because in our current world, the only things with human-level intelligence are humans. To understand ASI, we have to wrap our heads around the concept of something both smart and totally alien.

...So we’ve established that without very specific programming, an ASI system will be both amoral and obsessed with fulfilling its original programmed goal. This is where AI danger stems from. Because a rational agent will pursue its goal through the most efficient means, unless it has a reason not to.

...given the combination of obsessing over a goal, amorality, and the ability to easily outsmart humans, it seems that almost any AI will default to Unfriendly AI, unless carefully coded in the first place with this in mind. Unfortunately, while building a Friendly ANI is easy, building one that stays friendly when it becomes an ASI is hugely challenging, if not impossible.

If we program an AI with the goal of doing things that make us smile, after its takeoff, it may paralyze our facial muscles into permanent smiles. Program it to keep us safe, it may imprison us at home. Maybe we ask it to end all hunger, and it thinks “Easy one!” and just kills all humans. Or assign it the task of “Preserving life as much as possible,” and it kills all humans, since they kill more life on the planet than any other species.

...As for where the winds are pulling, there’s a lot more money to be made funding innovative new AI technology than there is in funding AI safety research…

...If ASI really does happen this century, and if the outcome of that is really as extreme—and permanent—as most experts think it will be, we have an enormous responsibility on our shoulders. The next million+ years of human lives are all quietly looking at us, hoping as hard as they can hope that we don’t mess this up. We have a chance to be the humans that gave all future humans the gift of life, and maybe even the gift of painless, everlasting life. Or we’ll be the people responsible for blowing it—for letting this incredibly special species, with its music and its art, its curiosity and its laughter, its endless discoveries and inventions, come to a sad and unceremonious end.

...people who understand superintelligent AI call it the last invention we’ll ever make—the last challenge we’ll ever face.

Read more here.