Evaluating UK natural hazards: the national risk assessment

Despite its relatively temperate climate and stable geography, natural hazards present multiple risks to human activity in the UK. These range from small-scale local occurrences, such as landslides, through regional incidents, such as flooding, to major high impact, low probability events, such as space weather. The impacts of such hazards can be wide-ranging but may include disruption to critical infrastructure and transport networks, detrimental effects on human welfare, and, in some cases, loss of life. This POSTbrief summarises the emergency planning for such hazards undertaken by Civil Contingencies Secretariat (CCS) of the Cabinet Office and published in the classified National Risk Assessment (NRA) and unclassified National Risk Register (NRR).


OVERVIEW
The UK is affected by natural hazards, such as flooding, heatwaves and wildfires, which can cause significant human, economic, environmental and infrastructure damage. Emergency planning for natural hazards is legislated for under the Civil Contingencies Act (2004), which puts the onus on local emergency responders to plan for civil emergencies. At a central government level, emergency planning is undertaken by the Civil Contingencies Secretariat (CCS) of the Cabinet Office and published in the classified National Risk Assessment (NRA) and unclassified National Risk Register (NRR).
The first stage in producing the NRA is identifying the natural hazards that could impact the UK. This is primarily undertaken within Government. Each hazard is 'owned' by a government department, which is responsible for identifying the 'reasonable worst-case scenario' for each hazard, and for determining its potential impact and likelihood. This permits comparison of each hazard type for prioritisation in emergency planning. Although the CCS issues risk owners with common guidance on the assessment procedure, methodologies for estimating impacts and likelihoods vary between departments, and hence between different hazards. The assessments undertaken by departmental risk owners are compiled by the CCS for publication in the NRA. Before publication, the NRA is scrutinised in an expert challenge process, which is mostly internalised within government.
The UK was one of the first countries to produce a national-level risk assessment and remains a world leader in this policy area. Despite this, commentators have identified several limitations in the current methodology, principally associated with: limited opportunities for bottomup (i.e. non-governmental) engagement; assessing the cumulative effects of multiple concurrent hazards; and the exclusion of long-term trends, such as climate change, from consideration. Alternative risk assessment methodologies have been developed in other sectors (such as utilities providers, the insurance industry and academia), which have been suggested as offering opportunities for future development of the NRA process.

Table 1. Summary of UK natural hazards included in public risk assessments
Comparison of natural hazards detailed in the Natural Hazard Partnership (NHP) Science Notes, 7 British Geological Survey (BGS) Geohazard notes, 8 Energy Technologies Institute (ETI) technical volumes, 9 the NRR 10

. Civil Contingencies Act (2004)
The Civil Contingencies Act (2004) was developed to address inadequacy in UK emergency planning. 11 It defines an emergency as: • an event or situation which threatens serious damage to human welfare in a place in the UK; • an event or situation which threatens serious damage to the environment of a place in the UK; or • war, or terrorism, which threatens serious damage to the security of the UK. 3 The Act specifies the roles and responsibilities of local responders for emergency contingency planning and civil protection. Two types of emergency responder are defined: Category 1 responders (e.g. local authorities, emergency services, NHS bodies and the Environment Agency) and Category 2 responders (e.g. utility companies, transport companies and the Health and Safety Executive Under this definition, the NRA is limited to 'acute' risks that relate to a single event or episode and require a one-time response. It is a broadranging assessment of the major emergencies that might affect UK citizens in the immediate term, 15 including unplanned hazards (such as natural hydrological, meteorological and geological events [ Table 1]; major accidents; and societal unrest) and threats with malicious intent (such as terrorism, Box 1). The current emergency planning doctrine is to plan for common consequences rather than individual risks, which informs the structure of the NRA. 16 Each type of risk in the NRA is 'owned' by a different government department, which is tasked with assessing its likelihood (i.e. the probability of an event exceeding a certain magnitude) 6 and impact. To inform the National Security Strategy and Strategic Defence and Security Review, since 2010, the CCS has separately undertaken the 5-yearly National Security Risk Assessment (NSRA), prioritising security concerns overseas that are unlikely to affect the civilian population living within the UK. 17,18 In 2019, the NRA and NSRA will be combined, with risks and threats (Box 1) assessed using a common methodology (see "Comparing risks", below).

Applications
The NRA has three main applications: • National resilience planning • Risk assessment by devolved administrations • Local-level emergency planning.
National resilience planning: The common consequences of all risks are drawn together at the end of the NRA to form approximately 20 emergency planning assumptions (previously published separately as the National Resilience Planning Assumptions). 16 The planning assumptions inform the UK Government's Resilience Capability Programme, which aims to improve civil emergency response (Box 1). Under the Resilience Capability Programme, government departments that 'own' the different capabilities (i.e. planning assumptions) are responsible for building resilience in those areas. 19 Risk assessment by devolved administrations: The NRA is used by the Scottish Government to inform the classified Scottish Risk Assessment (SRA). The SRA uses the NRA as an evidence base to begin identifying and assessing risks that are unique to Scotland or have significantly different impacts in Scotland than in the rest of the UK. 20 It follows the same methodology as the NRA to assess risk impacts and likelihoods, making the results of both assessments directly comparable, 20 and forms a bridge between the central government-level NRA and local-level emergency plans. Equivalent regional risk registers do not exist for England, Wales or Northern Ireland.
Local-level emergency planning: Local Resilience Forums (England and Wales), Regional Resilience Partnerships (Scotland) and Emergency Preparedness Groups (Northern Ireland) are collections of Category 1 and Category 2 responders, tasked with compiling Community Risk Registers (CCRs) in compliance with their statutory responsibility for local-level emergency planning (Box 2). The UK Government provides guidance detailing how risks should be evaluated so that CRRs are broadly comparable 16 and the NRA is used as part of the evidence base to develop these local risk assessments. 10 In some cases, local groups must identify and assess geographically-specific risks not included in the NRA, such as landslides. 21 These local groups typically do not include specialists from national organisations such as the British Geological Survey or Met Office and, in some cases, they have reported a lack of confidence in assessing risks that fall outside their areas of competence. 22 Devolution of power to city mayors has not included transferring responsibilities for emergency planning. 23

National Risk Register
Alongside the classified NRA, the Cabinet Office publish the unclassified National Risk Register (NRR), 10 which contains a synthesis of the information in the NRA. Due to the sensitivity of material, some NRA risks are grouped in the NRR and specific details, such as national vulnerabilities, are redacted. 24 The NRR is not intended to inform local emergency planning by Category 1 and Category 2 responders, but rather to inform the public, businesses and communities who do not have a legislative duty for emergency planning (Box 2); it advises on proportionate preparation and response. A recent study highlighted a lack of clarity regarding the intended purpose of the NRR, with some non-statutory local-level planners (who are not Category 1 or Category 2 responders and do not have clearance to access NRA) unaware of its existence or unconvinced of its usefulness because of an absence of actionable strategies. 25 Climate Change Risk Assessment 'Chronic' risks, such as climate change and antimicrobial resistance (POSTnote 595), which require an ongoing and/or policy response are not explicitly considered in the NRA. In the case of climate change, the risk is assessed by Defra in the UK Climate Change Risk Assessment (CCRA), 26 which is published on a five-yearly basis (since 2012) and looks ahead to 2080 and beyond. 27 The CCRA and NRA have different roles within government (i.e. one is for long-term adaptation planning and the other looks at short-term emergency response) and are not currently aligned with one another.

RISK IDENTIFICATION
The NRA identifies and defines the range of potential risks that might cause a national-scale civil emergency in the UK. Risks are included if they either appeared on the previous list (and nothing has changed in the interim) or if something new or different has been identified (either by a risk-owning department or the CCS). The NRA does not attempt to detail every possible hazard or threat scenario that could affect a significant part of the UK, but overall it tries to capture every possible impact. Hence, it aims to assess the UK's full capability to respond to civil emergencies on a broad scale (Box 1). The NRA excludes risks that are: too geographically localised and do not constitute a national emergency, such as landslides and sinkholes; 28,29 too low probability, such as near earth objects; 30 or for which the evidence base is insufficient for adequate characterisation at the present time, such as for subsea landslides breaking telecommunication cables (POSTnote 584; Table  1). 31 Adding new risks New risks are primarily identified internally, by consulting government policy experts who determine: • whether they constitute a civil emergency; • exceed the required impact and likelihood thresholds; or • have impacts that are distinct from the risks already on the register. 15 In some cases, external experts may highlight risks to the Government, 24 either by direct contact or via member organisations of the Natural Hazards Partnership (NHP; Box 3; Fig. 1). In rare cases, government departments might identify risks through more formal horizon scanning processes, such as Government Office for Science Foresight projects. 35 There is a possibility that risks will be overlooked if they fall between areas of expertise, are considered too complex, or if there is a lack of imagination in considering low probability events. These problems may be overcome through structured exercises or through consulting experts from a very wide range of disciplines. 36 Furthermore, even though risks are well-recognised by the scientific community, they could be overlooked in the NRA because of inadequate communication between the Government and academia. Commentators suggest this is a potential weakness in the NRA, causing it to act more responsively than predictively, for example where volcanic hazards were only added to the NRA after the 2010 eruption. 2

Risk assessment
For each identified hazard, the risk-owning government department is responsible for compiling evidence to inform the risk assessment process; for instance, Defra owns the UK coastal flooding risk. 37,38 A separate list of 'risks under review' is published alongside the NRA. These do not currently meet the impact and/or likelihood criteria for inclusion in the NRA or have an insufficient evidence base, but are still assigned a departmental owner and are re-assessed in the next NRA. 24,36 The NRA currently includes around 85 risks (roughly 50:50 hazards and threats; Box 1), plus around 40 risks under review. 24

Box 3. Natural Hazards Partnership (NHP)
The NHP is a consortium of 17 public bodies ( Fig. 1), which provides a forum for discussion and cooperation on natural hazards policy, science and service issues in the UK. It advises UK Government and devolved administrations on natural hazard-related issues.
The main NHP products are: • Daily Hazard Assessments: a daily overview of potential natural hazards that might affect the UK in the next 5 days. Hazards are mapped geographically and assigned impact and probability scores for assessment of their relative risks. The daily assessment is available to all emergency responders through ResilienceDirect. 39 • Science notes: scientific overviews of natural hazards that might impact the UK. These publicly-available notes contain more detail than is available in the NRR and also include more geographically-restricted hazards than would be included in the NRA. 7 • NRA consultancy: the NHP provides expert advice to help risk owners assess natural hazards in the NRA and NSRA, and is invited by the CCS to review and challenge the completed documents. 40 The NHP currently operates on a 'best endeavours' model and does not receive consistent funding or formal endorsement from the UK Government.
Commentators have raised concerns that, because member bodies' level of commitment is discretionary, there may be a risk that the NHP is vulnerable to public sector resourcing constraints. 41 A common set of assessment instructions is provided to risk owners by the CCS, but the assessment process varies between departments and between risks. Methodologies might include expert consultation, 42 analysis of historical datasets, or numerical modelling. 24 For example: • Wildfire risks are assessed by unstructured expert consultation • Volcanic hazards were initially assessed using structured exper consultation and numerical models, 43 and have since been changed by an expert committee • Coastal flooding is frequently re-evaluated using detailed statistical models. 44 The assessment of each risk category has three principal stages: • Definition of a 'reasonable worst-case' scenario.
• Quantification of risk impact.

Figure 1. The Natural Hazard Partnership wheel 41
List of the partner organisations (predominantly government departments and associated bodies) that comprise the NHP. Occasionally, the partnership also engages academics (e.g. NERC Knowledge Exchange Fellows).

Reasonable worst-case scenarios
There is a relationship between the impact and likelihood of different risks, with high magnitude events occurring less frequently than lower magnitude events. For example, major gales have greater impacts but occur less often than moderate winds. Practical emergency planning requires the identification of a threshold, above which the cost of allocating additional resources to prepare for the impacts of a hazard become unjustified  In practice, the relationship between impacts and likelihood can be difficult to assess, particularly given the diversity of hazard impacts considered in the NRA (see "Risk impacts and likelihoods", Table 2). Using a historical event makes it easier to assess the impacts from past records and to put a lower bound on likelihood, as it has happened. Where there is no useful historical event, other benchmarks for the reasonable worst-case scenario can be used, such as an event causing 100 or more immediate or eventual fatalities. 6 For some risks, the reasonable worst-case scenario is inherited and updated from previous NRAs, which is more efficient than creating a new scenario from scratch.
By planning for the reasonable worst case, the assumption is that emergency responders will be prepared to deal with other less severe occurrences of a particular hazard. 15 The NRA requires departments to think about variants of each risk and to quantify their maximum and minimum likelihood and impact -these do not receive the same level of assessment as the reasonable worst-case but represent the severity range. They are not published in the NRR. 10 If it is not possible to define a single reasonable worst-case for a particular hazard, and the impacts of multiple scenarios are distinct from other risks in the NRA, the CCS may define multiple risks within the same hazard category, such as river, coastal and groundwater flooding. Equally, if a hazard class has multiple manifestations but the impacts overlap with other risks on the NRA, they might be combined into a single risk category; for example, ash and gas hazards from volcanoes were combined between the 2015 and 2017 NRAs. 10,24 Risk impacts and likelihoods After identifying reasonable worst-case scenarios, the impacts and likelihoods of each hazard are quantified. Impacts are separated into defined categories and sub-categories (Table 2), which are scored between 1 and 5. 24 Each point on the scale represents a tenfold increase in the hazard impact. 45 These categories reflect the definition of a civil emergency in the Civil Contingencies Act (2004). The weighted average of these impact scores are then used to calculate the overall impact rating of each NRA risk. The weighting of each hazard class is decided by the CCS, along with relevant ministers, and is not made publicly available.
The likelihood of the worst-case natural hazards is also estimated on a coarse scale between 1 and 5, with a one-point increase representing a tenfold increase in the likelihood of occurrence. This is principally accomplished by analysing historical datasets and modelling past scenarios. 24,46 In each case, risk-owning departments are asked to detail the evidence used to support their risk assessment (published as an Annex to the NRA) and to specify their confidence in the evidence base used to calculate the likelihood and impact scores for each category.

Qualitative risk factors
As well as the quantitative assessment of impact and likelihood, the NRA contains sections for qualitative notes. These allow risk owners to detail multiple hazard scenarios and (in the 2019 NRA onwards, following amalgamation with the NSRA) long-term trends, including how risks will change due to climate change.

Comparing risks
The NRA, NRR and NSRA include combined risk matrices, which plot likelihood versus impact for each worst-case scenario together on the same graph, permitting direct comparison (Fig. 2). The NRA also contains a separate likelihood versus impact matrix for each type of natural hazard, which includes the reasonable worst-case, maximum and minimum scenarios. Currently, hazards (without malicious intent) and threats (with malicious intent; Box 1) are assessed using slightly different methodologies, preventing direct comparison of their likelihood and impact. 10 However, the 2019 NRA will implement a common methodology for hazards and threats, making the different types of national emergency more directly comparable.
Risk matrices are intended to distinguish hazard classes that have a high impact and high probability (priorities for emergency planning) from those that have low impacts and low probabilities (less concerning from a planning perspective). However, a recent study evaluating the NSRA suggested that they encourage under-prioritisation of risks that are unlikely to occur (low probability) but would have severe impacts if they did happen (high impact). 17

*Symbol Index
River flooding

NHP and academic scrutiny
The NRA review process has previously been subject to criticism. Both the Government Office for Science 36 and House of Commons Science and Technology Committee 2 have suggested that there should be more comprehensive scientific scrutiny of the risks, using a wide range of scientific, analytical and technical expertise. For natural hazards, scientific scrutiny primarily comes from a challenge group comprised of NHP partner organisations (behavioural science is reviewed separately by an independent panel; Box 3; Fig. 1). 38 This group reviews the scientific evidence used by risk owning departments to develop the risk scenarios and assesses the consistency of scores across the different hazard types. 47 However, new or emerging hazards might not have experts in the group, which can significantly weaken the process or provide no challenge.
The challenge process is organised by the CCS who vary the format depending on the type of external input that they require at different points in the NRA process. It may involve a period of reading ahead of time, followed by assessment and discussion in a roundtable meeting. However, because of the limited time frame for compiling the NRA, the challenge group may be called upon at relatively short notice. Given that the NHP operates on a 'best endeavours' model (Box 3), this can prevent some representatives from attending. Risk-owners or major contributors within the NHP can challenge their own risks and contribute to the debate. As the NHP comprises public sector organisations (Box 3), 39 academics are typically not involved in the challenge process and opportunities for academic scrutiny of the NRA are limited. The NHP can suggest specific reviewers from the member agencies if they feel that more scrutiny is required. However, the main opportunities for academic engagement are through direct interaction with risk-owning departments during the initial risk assessment or UKRI Knowledge Exchange Fellowships. These fellowships intermittently place academics within the CCS for months to years to provide direct academic input into the NRA. 48

LIMITATIONS OF THE NRA PROCESS Implementation
The UK was one of the first countries to adopt a national-level risk assessment 49 and is internationally regarded as a world-leader in this policy area. 12 However, since the initial establishment of the NRA, other countries have developed their own equivalent assessments with slight administrative and methodological differences. 49 In 2018, the Organisation for Economic Co-operation and Development published a report on National Risk Assessment: A Cross Country Perspective, which presents a comprehensive comparison of the national-level risk assessments compiled by 20 member countries. 40 Through this comparative process, the report identifies three limitations in the practical implementation of the UK approach: • Identifying risk-owners: In some cases it is difficult to identify lead government departments to act as risk-owners, particularly where the risk's impacts cut across siloed departmental remits. This could potentially to lead to risks falling between the cracks and going undetected or unassessed (e.g. volcanic ash before the 2010 Icelandic eruption).
• Balancing science and policy: The UK NRA process is governmentled and individual departments might over-or under-state risk severities to influence their prioritisation. The Government Office for Science (which is not a risk-owning department) and Government Chief Scientific Adviser have a key role in using scientific analysis to undertake independent moderation and arbitration of the process.
• Data availability: Recognising that opinions might differ regarding the nature of reasonable worst-case scenarios and the inherent uncertainties associated with likelihood and impact scores, the NRA risk scenarios are only assessed on a coarse scale. This accepts a level of uncertainty in the impact and likelihood estimates, and more detail may be required at a later stage for effective emergency planning and resource prioritisation. 50 Defining most probable scenarios Planning for reasonable worst-case scenarios can present challenges in communicating risk, as they are not necessarily the most informative scenarios for businesses and the public. In some cases they have been counterproductive. For example, during the 2009 pandemic influenza outbreak, communication of the reasonable worst-case scenario (that 65,000 people in the UK could die) led to inaccurate press coverage. 2 In addition, using extreme, but unlikely, scenarios and inadequately communicating more likely scenarios can be detrimental to trust in the risk assessment procedure. 36 Previous reports have suggested that the Government should communicate 'most probable' scenarios to better inform the public about the likely risks. However, despite the NRR being intended for a public audience rather than Category 1 and 2 responders with a statutory responsibility for emergency planning (Box 2), the risk information is based directly on the worst-case scenarios in the NRA. 2

Lack of bottom up engagement and trust
Stakeholders have emphasised the importance of trust in the NRA process so that it is considered to be a credible source of information. 36 However, a lack of opportunity for local-or community-level input in risk assessments can adversely impact trust. 25,51 The UK approach is more top-down (i.e. carried out by central government) than in many other countries, 50 with limited opportunity for bottom-up knowledge transfer. 25 Bottom-up community engagement may benefit the robustness of the risk assessment procedure; studies have shown that local communities can provide valuable geographic and physical data to validate risk scenarios. 52 Furthermore, engaging with communities can reveal improved methods of communication and highlight the specific information required by the public to drive locallevel resilience building. 53 Risk identification and assessment methodologies are always retained at a government level, but the UK NRA process is particularly internalised within government departments. Other countries use working groups of risk analysts that include experts from academia and the private sector to determine impact and likelihood scores, and to review their risk assessments, providing an independent validation of the methodology. 54 Additionally, some countries publicly release the scores for each impact category (i.e. Table 2) in each hazard class, as well as their confidence in the available evidence base. 55 Stakeholders believe that transparency, both in terms of the methods used and who is responsible for undertaking the risk assessment (e.g. which department), may increase trust and confidence in the NRA process, to the mutual benefit of emergency planners and analysts both within and outside of government. 25 Multiple hazards Natural hazard events may occur in clusters. This requires special consideration because their impacts can be more severe than if the hazards occurred individually. 56 For example, heavy rainfall might cause flooding that erodes sediment and triggers a landslide, 57 or heat-waves might cause drought that can lead to wildfires. 58 Methods for quantifying the impact and likelihood of multiple hazards are much less advanced than for individual events 56 and existing methodologies are typically based on theoretical hazard scenarios rather than real occurrences. 59,60 The few studies that have examined the impacts of real multiple hazard events have typically focused on only a limited range of their potential consequences. For example, an Environment Agency report into multiple hazards associated with Storm Desmond considered the economic and infrastructure impacts (such as to roads, flood defences and electrical substations), but did not look into other dimensions of the event (such as short-and long-term health impacts). 1 Multiple hazards are not fully integrated into the assessment of NRA risks and are typically overlooked in the development of reasonable worst-case scenarios. Currently, the NRA contains only a short section for risk-owning departments to note related risks and the NRR does not mention multiple hazard scenarios. 10 A more thorough qualitative evaluation of multiple hazard scenarios could be achieved relatively easily in the NRA using existing techniques, by creating tables of hazards that could potentially be related to the reasonable worst-case scenarios. 59,60 However, appropriate contingency planning for multiple hazards will likely require Cabinet Office to work between various risk-owning departments. 36 Long-term trends The NRA is primarily concerned with short-term emergency response to acute risks that could occur in the next few years. As such, it does not extensively detail the impacts of long-term trends, such as climate change and antimicrobial resistance, 10 instead containing a short qualitative statement on the potential for long-term variations in the likelihood or impact of different risk scenarios.
Statutory responsibilities for climate change adaptation are outlined in the Climate Change Act (2008) 61 and the risks are published independently in the CCRA. 26,27 This covers some of the same risk categories as the NRA (e.g. flooding, heat-waves, drought, wildfire), but considers average changes in climatic impacts as well as major events, looking ahead to 2100 and beyond. 26,27 The NRA and CCRA are not integrated due to the major differences in their risk assessment approaches (such as the timescales under consideration). Due to the differing statutory frameworks and intended users (primarily UK and devolved governments for the CCRA, compared with Category 1 and 2 responders for the NRA), the CCS and Committee on Climate Change (who advise the CCRA) do not routinely meet to discuss the two documents.
The absence of a co-ordinated risk assessment approach detailing how worst-case natural hazard risks will evolve causes confusion amongst some emergency planners. 25 In some cases, mitigation practices have long leadtimes and require adequate warning to implement. For example, wildfire risks will be exacerbated by climate change and the land management procedures necessary to mitigate its impact will take years to implement. 32 Therefore, reports have suggested that, although the Government needs to plan responses to emergencies in the immediate-term, it also needs to consider the longer-term (decadal) adaptation timescales required by some emergency responders, land use managers and regulators. 36 Some natural hazards that are defined as moderate risks are only addressed using generic emergency planning arrangements. Until hazards are considered high-risk, there may be no specific planning, including responder strategic documents and assessment. For example, wildfire has different capability needs to other fire incidents (POSTnote 603) but, as a moderate risk, it is dealt with by responders focused on structural firefighting and rescue approaches in Fire and Rescue Services' Integrated Risk Management Plans and Community Risk Registers. 32

OTHER RISK ASSESSMENT METHODS
Once a hazard is identified, accurate characterisation and communication of the potential risks and transparent quantification of the associated uncertainties can be used as evidence to inform decision-making. 62 As reasonable worst-case scenarios are inherently infrequent occurrences, determining their impacts and likelihoods using historical records may be limited by the absence of appropriate events. In these cases, risks may be characterised using statistical models or expert advice. 36 Methods have been developed outside of government to quantify and communicate low probability hazard characteristics and their associated uncertainties. These often go beyond the techniques current employed in the NRA and previous reports have suggested that they could be incorporated in the future. 36 Environmental statistical analysis Utility companies have a statutory and regulatory responsibility for assessing the risks posed by natural hazards and to build infrastructure with the resilience to withstand events up to a certain frequency (for example, buildings are typically designed to withstand events that occur much less than 1 in every 10,000 years). 9 Similarly, insurance companies and other financial operators depend on accurate quantification of risk uncertainties to ensure that they have sufficient capital to pay out in case of a particular magnitude of hazard occurring. 63 To assess these risks, private operators undertake environmental statistical analysis, which allows them to characterise the relationships between natural hazard frequencies and impacts, such as how impacts increase with time, and to quantify uncertainties in these predictions. 9 In the global public sector, environmental statistical modelling is used widely to assess economic loss, as well as in flood, drought and extreme weather analysis. 64 However, collaboration between the public and private sectors has been limited by: • Inconsistency in compiling and sharing the required datasets of past loss.
• An absence of national-scale modelling tools. 65 • The black box nature of most commercial catastrophe models. 39 Recent publications detailing the extreme value statistical techniques used by utilities operators 9 and the release of open source catastrophe models suggests the possibility of more extensive use of these models to assess public sector risk by the Government and academics in future. 66 At present, NRA risks with the most extensive probabilistic hazard assessment are flooding 44 and meteorological events (storms, low temperatures, heatwaves), 67 which use existing Environment Agency and Met Office models, respectively. The Met Office has recently initiated collaborations with the insurance sector 68 and a Defra-led reinsurance scheme (Flood Re) employs industrial catastrophe modelling to provide insurance cover to UK homes at high risk of flooding. 69,70 Data and methodologies Many sectors use statistical methods to determine relationships between the magnitude and frequency of natural hazards. In some cases these are based on extrapolating from historic observations alone, such as extreme value analysis. 9,71 In other cases, they use historical data to inform model simulations 72,73 or complex numerical modelling to randomly generate thousands of "hypothetical" hazard events (stochastically-generated synthetic event catalogues). [74][75][76] These approaches can be used extensively to assess the characteristics of low frequency natural hazard risks and allow quantification of the uncertainties associated with hazard predictions. 9,74 Different sectors then use a range of models to translate the calculated event characteristics into impact, allowing for quantification of loss (such as financial cost, infrastructure damage or human injury) over a given return period. 63,77 Some of the observational data used in these techniques are proprietary. However, private sector companies often use publicly-available datasets, including meteorological and geological data, many of which are produced by governmental organisations such as the Environment Agency. 78 Using historical data alone to inform statistical methods risks missing the lowest frequency and highest impact events. Using environmental statistical techniques may also produce inaccurate predictions when the magnitude and frequency of hazards is changing due to long-term trends, such as climate change. 9,79 However, it is possible to simulate alternative past climates to understand hazards without thinking about climate change 73 and more recent models are beginning to integrate climate change information into their statistical predictions. 80,81 Structured expert elicitation Expert consultation in natural hazard risk assessment typically takes the form of an 'unstructured' committee of specialists, who are asked to produce a consensus opinion on hazard scenarios, either through discussion or a vote. 82 The credibility of their judgements relies on the panel being composed of qualified and informed experts from an appropriate range of relevant disciplines and experience. 83 However, studies have suggested problems with this unstructured elicitation: • In a situation where observations are scarce, it is unlikely that experts will ever entirely agree, forcing a consensus of opinion that risks missing important or nuanced details.
• There is no way to formally quantify the uncertainty associated with these expert predictions. 82 Even with the best possible expert selection, tests show that amongst specialists there is a poor correlation between the level of an individual's perceived authority and their ability to make accurate hazard judgements. 84 Some of these problems can be addressed using a more structured expert elicitation method, involving a formalised procedure for pooling a diverse range of expert opinions. 85 Although a number of procedures have been developed, there is evidence that the 'Cooke method' can improve expert estimates. 82,83,87 In the Cooke method, a panel of experts are asked to provide best estimates for target risk assessment questions (for example, the length of time that a dam can withstand a leak), 85 along with associated uncertainty ranges. The experts are also asked to estimate a series of parameters with associated uncertainty ranges for related values that are known to the facilitator. The experts' judgements for the target questions are then weighted according to the statistical accuracy and informativeness of their estimates measured against the known parameters. 82 This method can be implemented in a manner that minimises influences on individual scientists, such as peer or institutional pressure, thereby promoting neutral, evidence-based and informative quantification of scientific or risk uncertainties (Fig. 3). 85 The Cooke method has been used to quantify risk and uncertainty in a diverse range of fields, including engineering, 84 ecology, 88 biomedicine, 89 and finance. 90 For natural hazards, it has been used to quantify the uncertainty on climate change estimates, 91 earthquakes 92 and volcanic eruptions. 42 For example, it has been used repeatedly from 1995 up to the present during the volcanic crisis in the UK Overseas Territory of Montserrat to forecast the volcano's eruptive behaviour and provide real-time decision support for civil protection. 93 To date, the use of structured expert elicitation in the NRA has been limited. This is due in part to the time and cost of the exercises, which can take 2 to 4 weeks to complete (with analysis and reporting), cost a few to several thousand pounds and require some time commitment from the expert panel. 94 However, structured expert elicitation has been used to improve the NRA evidence base for volcanic gas hazards. 43 Characterising risks Risks in the NRA are characterised using two measurements: their likelihood and impact (Fig. 2). 10 However, more sophisticated methods to describe the nature of risks have been developed using additional risk attributes (Table 3, Fig. 4). 95,96 The Government Office for Science Blackett Review of High Impact Low Probability Risks recommended that the Government consider using such an approach in the NRA process, 36 specifically citing a method based on concept work by the German Advisory Council on Global Change. 97,98 This characterises risks based on nine criteria: (1) damage; (2) probability of occurrence; (3) incertitude (indicator of different uncertainty components); (4) ubiquity (geographic extent); (5) persistence; (6) reversibility; (7) delay effect; (8) violation of equity; and (9) potential of mobilisation (social unrest and psychological reactions). Although many potential risk types could be defined using a combination of these criteria, six 'genuine' classes are specified (each given a name from Greek mythology), which cover extreme characteristics and are most useful for risk management (Table 3; Fig. 4). 97 Defining the characteristics of risks in this way allows for more efficient hazard management; each risk type can be dealt with using a specific policy approach. 98 For example, one risk might require more scientific research, while others might require a precautionary approach or a focus on communication. 36 Apart from limited use in Environment Agency flood risk management, this type of multi-criteria risk characterisation remains largely unused in UK emergency planning. 95 Detailed risk characterisation is not included in the most recent NRA or NRR, 10 but this type of classification would aid in broadly highlighting actionable strategies, as required by emergency planners. 25 Diverse and conditional advice To justify and aid in decision-making, policy-makers sometimes pressure experts to express risk characteristics using definitive quantitative descriptions. 99,100 Research has shown that carefully selected scientists, who attempt to incorporate probability into their judgements, provide the best predictions of technical problems. 101 The Cooke method (see "Structured expert elicitation", above) further improves on individual performance by aggregating estimates and providing a quantitative measure of uncertainty. 82,83,87 However, social scientists have suggested that by asking experts to reduce the data and ideas behind their predictions, policy-makers can obtain an oversimplified picture of complex phenomena and may not appreciate any potential ambiguity or ignorance (the unknown unknowns) in the stated confidence range. 99,102,103 They question the validity of quantifying uncertainty by reducing or aggregating it to a single number. Instead, they suggest that experts should be allowed to provide pluralistic and conditional advice; for example, "under condition A I would expect outcome X but under condition B I would expect outcome Y". This is particularly salient when scientific advice is more oriented towards action than prediction, 99 and sensitivity analysis and interval analysis can be used for quantification as part of such plural conditional approaches. Experts that give conditional advice might not provide more accurate estimates, 104 but this would allow more informed policy debates and increase accountability and trust in expert predictions. 99 In the NRA, risk-owning departments are asked to provide a qualitative statement on their confidence in the evidence base used to undertake the risk assessment, and the impacts and likelihoods of reasonable worst-case scenarios are only quantified on a coarse scale (where an increase of one point represents a tenfold increase in impact or likelihood) in the hope that all uncertainty in the scientific evidence falls within this broad range. 50 However, as the NRA requires each hazard to be defined in only a single reasonable worst-case scenario, 15 experts are unable to provide pluralistic and conditional hazard advice; they might know that a given reasonable worst-case scenario under-or overestimates the real potential hazard under certain circumstances, but are unable to communicate this to decisionmakers. Additionally, experts are unable to convey whether their selected reasonable worst-case was contentious. Allowing scientists to share the complexity and limits of their predictions would permit more informed science-based policy-making, improve democratic accountability and highlight the reasons behind risk management decisions, increasing trust and credibility in the risk assessment process. 99,100,103