Different Lenses to Prioritize AI Risk

Source: DALL-E-3

AI is multi-faceted – it can create new artworks and proteins as well as drive cars. Naturally therefore, AI risk is also multi-faceted. We can organize it in different ways. This earlier post uses four high-level categories. This new NIST framework has 12 categories (and is still not comprehensive). At the end of the day, these are all important risks, and as a society, we need to keep track of and manage all of them. But at any given point in time, it can be useful to have a way to narrow in and focus only on a subset. In order to do this, we can apply different prioritization lenses.

One lens people often use is velocity, i.e. how fast is the risk coming at us, or how close in time the foreseen risk event is. This is a lens used by many in the AI bias and ethics camp. They argue that we should deal with the AI risks that are coming at us soon, or that are here already, rather than expend our efforts on AI risks that are further away.

Another lens often used is optionality, i.e. how much freedom of movement do we have in managing the risk. This is a lens often used by people in the existential risk camp. Risk events that can extinguish most of humanity naturally leave us with much less optionality than a risk event that “merely” risk costing us billions of dollars. This is similar to the cliched-but-true expression that there is no planet B when it comes to climate change.

I would like to make an argument for using the lens of uncertainty. Uncertainty is fundamental to risk. In fact, some definitions equate risk and uncertainty, defining risk as just the presence of uncertainty. This leads to the somewhat unintuitive definition used by e.g. ISO, of risk simply being a deviation from expected, whether positive or negative. Following Hubbard, I prefer to use the word risk in the sense that is more akin to its common usage, as the possibility of an adverse outcome. But however defined, uncertainty is at the core of risk. If the arrow of time could be reversed at the level of human perception and we had equal knowledge of the future like with the past, there would be no uncertainty and there would be no risk.

If we use uncertainty as the lens, we can see that some AI risks have higher uncertainty than others. For some of the misuse risks, like biological weapons, the effects of adding AI to the risk picture make the potential risk events worse – both more severe and more likely – but we can still roughly foresee what a risk event will look like. Similarly for inequality risk, we know AI will make inequality worse since it is trained on datasets that have human biases built-in, and since LLMs are a scale business where rewards go to capital rather than labor. So we can roughly see the contours of what this might look like.

However, there are some risks from AI where there is much greater uncertainty and we can truly not foresee the consequences of risk events. This is especially true for risks that strike at the core of how society works and how humans work, such as disinformation and automation.

For society, a core part of what makes a democracy work is the flow of information. The premise of handing power to elected leaders hinges on an informed electorate choosing their leaders wisely. At least that has been the accepted opinion since the invention of recent democracy. With AI, it seems that we are on the path toward a world where we will no longer know what is truth and what is fiction, given all the deepfakes in circulation. This has the potential of seriously undermining democracy.

We have yet to see how much this risk will actually play out, there are also arguments that the risk is overblown and that societies have always dealt with disinformation in some form or other. In any case, it is a risk where the uncertainty is great, given that we really can’t foresee the effects on democracy if all information is mistrusted until proven correct.

This week, interesting news in the disinformation risk space included the news that X will have a new AI-powered feature, where Grok summarizes user comments on news. That seems like it can be deleterious for our shared epistemics. We also saw how AI-generated text is being used more and more in scientific papers. If the papers’ authors are leaving the line “certainly, here is a possible introduction for your topic” in the paper, it is unlikely they have checked the rest of the paper for accuracy. We also saw a new paper from Google DeepMind describing how LLMs are getting better at manipulative persuasion. There has been a lot of speculation regarding the use of disinformation in the upcoming US election, but now we also see increased use of deepfakes in India. We don’t know yet if seeing Modi dancing Gangnam Style will impact the election, but clearly disinformation is a risk that will only become more important, especially as more capable open-source LLMs spread.

For humanity, much of what we call meaning in life is derived from the work we do for a living. So similarly here, we have very little insight into how people would choose to live their lives if and when much of their work gets automated by AI. We see arguments both arguing that AI automation is likely to remove all need for human work as well as the opposite who argue that there will always be new jobs as there has been in the past. But given that AGI by definition is a machine that can do anything a human can do, more efficiently and more cheaply, as Tegmark points out in his essay in Possible Minds, it seems unclear what those jobs could be.

In general, we have very little foresight into how likely automation is and when it would take place for common human jobs. Before we had GenAI, the common thing to say was that machines could not replace white-collar jobs, only blue-collar jobs. This changed overnight with GenAI. A McKinsey study from last year showed how, pre-GenAI, the expected timing of when AI could reach human level varied from 2030 to 2070. Then suddenly, as can be seen in the figure below, post-GenAI, most dropped to 2025-2035.

Source: McKinsey

So we should not take anything for granted. Currently, humans still have a decisive advantage when it comes to navigating in the physical world. But if a breakthrough like that with transformers in reinforcement learning happens in robotics, that could quickly change. Robotics is still hampered by the lack of data.

Two other areas of human advantage holdouts are skills like creativity and empathy. We should therefore pay extra attention to when we see examples of AI making inroads for those elements. This week we saw news on some of these. There was a Harvard study showing that GPT-4 performed better on empathy than humans, and there was a study suggesting a creative process in LLMs that is quite different from that of humans. Overall, this suggests an update toward automation risk becoming more important. At the same time, it’s important that we remember that these studies only reflect slivers of reality. As we are reminded in this piece, performance on tests do not equal performance in the real world.