The Current State of Societal AI Risk Management
Image: GPT-4. Not sure what it’s trying to write top left, but I like having risk and ethics on each side.
The risks from AI need to be managed at several different levels, by different groups of people. At the company level, company executives need to manage the risks from AI to their organization. At the geopolitical level, country leaders need to manage the risks from AI in inter-country power struggles.
However, I would argue that the most important level is AI risks to society. AI poses large risks to society as a whole, which are likely to have an impact on humanity in the longer run. At the societal level, it seems very likely we will see tremendous benefits from AI, but we are also facing an unprecedented set of risks. This is what needs to be tackled through what I call “Societal AI Risk Management”.
Not all of AI risks are new and many of them have earlier analogues, but given that AI is crystallized intelligence and intelligence underlies all aspects of society, AI arguably poses the largest set of new risks to society that we have ever had to deal with. These are risks that need to be managed by society as a whole, by a combination of government, companies, civil society and academia.
I therefore plan to start doing regular short posts taking stock of recent events in the AI space and interpreting what they mean for the overall picture of Societal AI Risk Management.
Before we get into recent events, we need to set a few definitions and frameworks:
What are the AI risks at the societal level? There is no one overall framework for AI risk, but a good way to organize them is that, at the society level, there are four categories of AI risks.
First, misuse risk – this is the risk that people will misuse the capabilities of AI models for malicious purposes. This category includes the risks of people using AI to create weapons, or to create disinformation. The weapons that tend to be most commonly discussed are biological weapons and cyberattacks, since we can already now clearly envision how AI can be used nefariously for these types of weapons.
Second, structural risk – this is the risk that AI will lead to large structural changes in society. This category includes the risk of AI-induced mass unemployment and the risk of AI-enabled dictatorships.
Third, inequality risk – this is the risk that AI will cement and exacerbate inequality patterns in society. This category includes the risks of biases and discrimination.
Fourth, misalignment risk – this is the risk that the AI models themselves will do something that is detrimental to humans or society. This category includes the risks of unintended actions by an AI seeking to complete the goals it has been given and the risks of side effects from interactions between multiple AIs (colluding and/or competing).
What is societal AI risk management?
Each entity (a company, a person, etc.) has certain objectives, but the inherent uncertainty of the world means that those objectives may not be met. Risk management simply means managing that uncertainty as well as possible in order to maximize the likelihood that the objectives are met. At a societal level, risk management therefore means maximizing the likelihood that society achieves its objectives and minimizing downside surprises. The objectives of society are of course multi-faceted and complex, but could perhaps be summarized as progress on the level of society as a whole and health and safety for its individual citizens. So societal AI risk management basically means maximizing the likelihood that society has a flourishing future given the introduction of new sources of intelligence into society.
What are the options for societal AI risk management?
In practice, there are some choices when it comes to managing risks. At the highest level, risks can be avoided, transferred, mitigated or accepted. Unfortunately, for AI, three out of four of those options are not available. It is impossible to avoid the risks since it seems impossible to stop the technological evolution, there is no insurer at the societal level that has a planet B to offer if things go wrong, and the risks are too large to be accepted wholesale. So mitigation it is.
How does societal AI risk management work in practice?
Risk is a combination of likelihood (how likely is a risk event to occur) and impact (what are the consequences if the risk event occurs). We can mitigate risks through either of those channels – we can lower the likelihood that things go wrong and we can blunt the impact if things do go wrong.
If you picture the objectives of society as the goal we want to head towards, the north on our compass, we can see risk management as keeping the needle pointed towards north. We can then analyze events as hits to the needle, making it deviate from north. Hits can be large or small and they can be long-lasting or short-lived.
So to begin with, let’s look at three recent events and see what impact they may have had on the needle.
Meta just released Llama 3. This is now the most advanced open-source model available. The debate around whether or not to open source models is an unsettled one. Likely, it is still ok to open source models at this level of capabilities, but it would be unwise to open source models at higher levels. It could arguably also be beneficial to open source models at this capability level, since it allows for more research into how understanding how the models work. But the concern is that the debate over open source becomes so highly charged and the positions so cemented that when we get to more powerful models, some people will have a strong bias to open source them regardless. The positive news regarding Llama 3 is that Meta is apparently still deliberating whether to open source the most powerful version of Llama 3, the 400B version, which is still being trained. But since Mark Zuckerberg recently said he would open source all the way to AGI, there is perhaps not too much comfort to take from this.
On balance, this is likely an increase in risk given that we will likely soon have an open source model that is getting closer to the levels of GPT-4.
AI can now prevail in (simulated) dogfights, as per the US Air Force. This is a milestone for the use of autonomous AI in military operations. The use of AI for military purposes is, like most other use cases, a dual-use technology. The use of AI could mean less human casualties in conflicts and it could mean greater deterrence. However, the specter of lethal autonomous weapons system (LAWS) also looms large with every upgrade of AI’s military capabilities.
On balance, this is likely a small increase in risk since it will increase the likelihood of AI being used in conflicts.
Churn in the Superalignment team at OpenAI. The Superalignment team is their team tasked with managing the risks from superintelligence, i.e. when an AI is smarter than humans. This team has had some churn recently. Two people – Leopold Aschenbrenner and Pavel Izmailov – were supposedly fired for allegedly leaking information. Then, another person, Daniel Kokotajlo, supposedly resigned. Very little is known about these events, but they seem plausibly bad from a societal AI risk management perspective. First, these were accomplished researchers that the Superalignment team will now be without. Second, these were also researchers highly attuned to the risks of AI. Third, it might suggest a heightened focus on secrecy at OpenAI, which could be bad from a transparency point of view.
Overall, a small update, but one towards potentially increased risk, since OpenAI is still in the lead when it comes to AI development and there is a lot riding on the Superalignment team.