Nick Bostrom was recently awarded a €2 million ERC Advanced Grant, widely considered to be the most prestigious grant available from the European Research Council. The grant will allow Nick Bostrom and a team of FHI researchers to continue their work on existential risk and crucial considerations.
The title of the grant is “UnPrEDICT: Uncertainty and Precaution—Ethical Decisions Involving Catastrophic Threats.” The abstract is as follows:
The goal of this project is to supplement or replace the precautionary principle with decision guidance that better handles both normative and empirical uncertainty in contexts of speculative but potentially catastrophic consequences. It has been claimed that emerging technologies such as geoengineering, biotechnology, or machine intelligence could have catastrophic impacts on human civilization or the biosphere, indicating the need for precaution until scientific uncertainty has been resolved. Yet it is unclear how to apply the precautionary principle to cases where the deeper investigations of scientific uncertainties that it calls for can themselves be a source of catastrophic risk (in e.g. gain-of-function research and geoengineering experiments). Furthermore, the precautionary principle fails to account for moral uncertainty—even though many decisions depend more sensitively on ethical parameters (e.g. our obligations to future generations, discount rates, intrinsic value of nature) than on remaining scientific uncertainties.
Building on recent advances in decision theory, computational modelling, and domain-specific risk assessment techniques, and using tools of analytic and moral philosophy, this project will: (1) develop methods that account for normative uncertainty by combining voting theory with ongoing work on moral uncertainty, identifying parameters that make the largest practical difference; (2) combine these with methods for dealing with sources of empirical ignorance (such as information hazards, anthropic shadows, model uncertainties) into a theoretically well-motivated framework that comprises normative and empirical uncertainty; (3) derive mid-level principles and procedures from this theoretical framework by working through three case studies (geoengineering, dual-use biotechnology, and automation and machine intelligence) offering better practical guidance on speculative but potentially catastrophic risks than does the precautionary principle.