Understanding Rogue AI: Impact, Neutralization, and Prevention

As the advancement of?artificial intelligence?(AI) models keeps evolving steadily, there’s a growing concern that they may eventually surpass our human limits. While this evolution isn’t necessarily dangerous on its own (they’re, in fact, superior to us in many aspects, starting from their computational power), experts in the field warn about the threat posed by “evil” AI.?

Highly-intelligent AI systems can be used for many purposes, but what if they become so intelligent as to rebel against their creators? Suppose some of them become the much-dreaded rogue AIs. In that case, they can inflict severe damage on our society, starting by defeating all current human-created cybersecurity measures.

But what is a Rogue AI? Is it the Skynet-level, destroy-all-humanity existential threat that many doomsayers talk about, or can it be subtler yet perilous? Like many other talking points around AI, reality is strictly intertwined with myth, hype, and misunderstandings.?

Let’s try to untangle this knot to rationally assess how much our society is threatened by (real or hypothetical) rogue AIs.

What Is a Rogue AI?

First, let’s look at what “rogue” actually means and how this word can be applied to AI.

According to the Collins Dictionary, a rogue is:

• A wandering beggar or tramp; vagabond
• A rascal; scoundrel
• A fun-loving, mischievous person
• An elephant or other animal that wanders apart from the herd and is fierce and wild
• An individual varying markedly from the standard, esp. an inferior one

At least the Collins Dictionary hit the bullseye in identifying rogue AIs as the elephant in the room. Arguably, the correct definition of a rogue AI is a mix of all the ones described above, especially the part that describes it as “an individual varying markedly from the standard.”?AIs are created with one purpose in mind: to serve humanity. What defines an AI as a rogue is the moment it stops doing that, and either poses a threat to us or starts serving its own purposes or goals.

An infamous example of an AI going rogue is Tay, a chatbot developed by Microsoft?to entertain Twitter users back in 2016 with witty jokes and comments. Just a few hours after its launch, a group of trolls from 4chan re-trained the AI with racial slurs so that it rapidly started spouting vulgar, anti-semitic, misogynist, and racist comments.

In less than one day, the AI had to be shut down.

AIs can become rogue in several ways:

1. When someone tampers with them with malicious intent, especially during its early stages (as in Tay’s example above);
2. When they are inherently dangerous (think of military-grade AI created for warfare purposes), yet not adequately overseen during their early stages, and eventually grow out of control later on;
3. When someone purposefully builds them to be evil, dangerous, or destructive;
4. When it becomes sufficiently autonomous to set its own goals, these goals do not align anymore with humanity’s well-being (or their creator’s will).

The fourth option is unlikely (at least today), as it requires a degree of self-awareness that is still very far from actual AIs capabilities.

The other three, however, are not.

What Could the Potential Impact of Rogue AI Be?

Before delving into what we could do to prevent AI from going rogue, assessing what kind of damage they could do is essential. In Tay’s example above, the impact is essentially negligible, but this comes from two main reasons:

1. The purpose and capabilities of Tay were relatively harmless since the beginning;
2. Microsoft’s damage control reaction was swift.

However, in different scenarios, the impact of a rogue AI can be much more devastating. An example is AI purposefully created to exploit cybersecurity vulnerabilities for stress tests or cyber warfare purposes. If (or when) they grow out of control, they can shut down entire networks critical to our society’s correct functioning (such as energy grids or healthcare systems). Dangerous actors who care little about their safety or controllability, such as cyber-mercenaries or hacker conglomerates, can develop AIs with a high potential of going rogue.

Rogue AI can also become dangerous when they are entrusted with significant responsibilities. Models that have not been overseen correctly can eventually make wrong assumptions in particularly delicate fields, such as the oil and gas industry or automated warfare. A potentially rogue AI could be generated simply by designing an AI agent that is too intelligent for its good.

For example, a military AI whose goal is to neutralize the IT infrastructure of the enemy may figure out that the best strategy to achieve its human-set goal is to define its sub-set of goals. One of these goals may involve obtaining extra data from enemy humans by shutting down some of their hospitals, aqueducts, or other essential infrastructures, causing unintended damage to the civilian population.

What Can We Do to Prevent (and Neutralize) Rogue AI?

The extent of the threat posed by rogue AI has been taken very seriously by most actors at the forefront of the AI revolution. For example, OpenAI, the creator of ChatGPT, recently announced the establishment of a team of top AI experts who will work on a project known as Superalignment. The purpose is to build a “roughly human-level automated alignment researcher” to conduct safety checks on superintelligent AIs and keep them under control.

Others,?like UNESCO, suggested ethical rules and frameworks for AI governance. To prevent, or at least minimize the risk of an AI going rogue, all companies who develop AI models must adhere to ethical standards that include critical points, such as:

• Ensure the physical and digital security of humans and their privacy;
• Incorporate ethics in their design so that no bias is incorporated;
• Be transparent about the functions, purposes, and limitations of the algorithms;
• Provide full disclosure to clients and users about how the AI works;
• Oversee the design and training of the AI through all its stages of development and keep monitoring them even after they’re released into the real world;
• Make sure humans will always remain in charge and can shut the entire system down whenever necessary.

Besides what companies can do, it’s also essential to establish global policies that acknowledge the risk posed by rogue AIs and agree on international agreements and treaties to prevent them from becoming a threat. Policymakers are responsible for deciding what’s best to protect the public without halting AI research and development.

Similar to the negotiations about the ban on the use of nuclear weapons arising from a generalized fear of nuclear Armageddon, opposing countries should strive to find common ground to establish what can and what cannot be done with AI to avoid catastrophic outcomes.

The Bottom Line

It seems we’re still far from the apocalyptic scenarios depicted in sci-fi movies where humanity is on the verge of extinction because some AI went rogue. Still, now it is the right moment to prevent this risk from ever happening.

Today, we are responsible for establishing the ethical and rational rules that will steer the future of AI research to pave the way to a better tomorrow. Like other world-defining scientific revolutions in the last few years, such as genetics and nuclear energy, AI it’s neither good nor evil per se: its threats come from the uses we will make of it.

And it’s our duty as humans to establish the moral foundations of what’s good and what’s evil to better control this new force of nature that we just created.