The UK government must act now to ensure the UK is prepared for future extreme risks even greater than Covid-19, a new report finds.
Disease warfare, leaks of dangerous pathogens from high-security labs, and misuse of artificial intelligence (AI) systems are identified as key threats to the UK and the wider world. The report, Future Proof, is co-authored by FHI’s Toby Ord and the Centre for Long-Term Resilience and features contributions from FHI’s Cassidy Nelson, Gregory Lewis and Markus Anderljung. It analyses some of the most extreme risks the UK faces, rates the government’s current level of focus on these threats, and provides a costed roadmap for tackling these risks in the upcoming Comprehensive Spending Review.
The report identifies a number of extreme risks to the UK and the wider world, including:
Biological weapons and lab leaks. Human-caused pandemics, either by an accidental lab leak or by deliberate biological attack, are identified as even more of a concern than naturally occurring pandemics.
Misuse of AI systems. As AI becomes integrated into safety-critical systems — from self-driving cars to military equipment — it raises the stakes of accidents, malicious use of this technology, or AI systems behaving in unexpected ways.
Toby Ord, one of the report’s lead authors and a Senior Research Fellow at Oxford University’s Future of Humanity Institute, said:
“Extreme risks define our time. By my estimate, the likelihood of the world experiencing an existential catastrophe over the next one hundred years is one in six — Russian Roulette. We cannot survive many centuries operating at a level of extreme risk like this. The government needs to make serious efforts now to increase our resilience to these threats. Future Proof provides a roadmap to do just that.”
Future Proof finds that extreme risks do not get the attention from the UK government that their severity demands, and that they tend to be excluded from the government’s risk assessments. The report also identifies a tendency for the government to ‘prepare to fight the last war’ and focus only on the types of disasters that have occurred in the past. Covid-19 was a good example of this: pandemic preparedness planning prior to Covid-19 presumed an influenza outbreak, so the government was wrong-footed when confronted by a coronavirus. The report identifies a rare window of opportunity for the UK to transform its
resilience to extreme risks.
Angus Mercer, one of the report’s lead authors and the Chief Executive of the Centre for Long-Term Resilience, said:
“As the UK builds back from Covid-19, we have a once-in-a-generation opportunity to transform its resilience to the most extreme risks we face — especially risks from pandemics and AI. The government has already committed to producing an AI strategy, a National Resilience Strategy and a biosecurity review. It must put resilience to extreme risks at the very heart of these new policies.”
Future Proof provides a roadmap for the UK government to transform its resilience to extreme risks. Key recommendations include:
- Increased caution before undertaking ‘gain of function’ research, which aims to increase the virulence and lethality of pathogens and viruses.
- Ensuring that extreme risks become much more central to the UK government’s risk management processes.
- Setting up a ‘three lines of defence’ approach to risk management in the UK, led by a new Chief Risk Officer.
- Launching a new £3 million prize to incentivise the development of clinical metagenomics, a ‘game-changing’ new technology to help identify future pathogens early in an outbreak.
- Establishing a new National Centre for Biosecurity, tasked with ensuring preparedness for the full range of biological threats the UK faces.
- Boosting UK funding for technical AI safety research, to help avoid the dangers of unsafe AI systems.
Sophie Dannreuther, one of the report’s lead authors and a Director at the Centre for Long-Term Resilience, said:
“We do not know which extreme risk event will come next — it might be another pandemic, or it might be something completely different. But we do know what many of these extreme risks are, and how best to prepare for them. The experts in this field have a clear message to the government — the UK needs to act now, and they are here to help.”