This academic research project is done in part of EPA1361 Model-Based Decision-Making course. We used the Flood Risk Management Plan on the Ijssel River for the case study. It involves multiple stakeholders that share different perspectives and stand on different administration levels. Every group took a role and we played as Dike Ring 1 & 2 Analyst.

A model was shared so students have the same starting point. It was used to experience how to use model against deep uncertainty and how to utilize model as discussion facilitator instead of the holy grail. Aside from building the model, there is a serious game consist of two rounds of debate illustrating how decision-making process happens in real life.

This article is written based on assignment that is submitted.


The Netherlands is a country that has been battling water for centuries (van der Brugge, Rotmans and Loorbach, 2005). The Dutch are threatened by water on two fronts, the sea and major European rivers. River IJssel is one of the problematic rivers. The river IJssel originates from the River Rhine which transports runoff water from the Alps (van Urk, 1978) and creates an “a melting-water/rain-fed river” (Waterman, Misdorp and Mol, 1998, p.115). The river is relatively small compared to other rivers in the Netherlands. Thus, the height of the river varies over the year. These explain floodplains surrounding the river experiences flooding on a regular basis (Venterink, Wigeman, van der Lee and Vermaat, 2003).

The river IJssel meanders through two Dutch provinces: Gelderland and Overijssel. The upstream part of the river IJssel is located in the Province of Gelderland, the downstream part (that flows into the IJsselmeer) is part of Overijssel. River IJssel floods both provinces regularly. As it’s not possible to keep heightening the dike, a joint project between provinces are created to benefit all of them instead of preventing the flood individually. The dikes along the river are divided into groups called Dike Ring 1 until 5. Three of them are located in the Province of Gelderland and the other two are located in the Province of Overijssel. Beside province stakeholders, the Dike Ring stakeholder is also involved in the decision-making process.

Figure 1: Map of Ijssel River part in the Netherlands (Ciullo, n.d., p.11)

This project is called the Room for the River Project. “The Room for the River Directive policy line was created to consider water as a structuring principle for spatial development” (Rijke, van Herk, Zevenbergen & Ashley, 2012, p.370). This project contains: dike heightening, deepening the floodplains, creating by-passes (Rijke et al, 2012). The objectives of the directive are: increasing safety, and improving the areas surrounding the river. (Rijke et al, 2012). A model is used to explore the most effective ways to implement the measures.

As part of the whole serious games process, this analysis is done from the perspective of an analyst group for Dike Ring 1 & 2. Dike Ring 1 & 2 want to maintain their modern style farmlands beside protecting their land against flooding. A research question is formed to be the goal of this research.

How to minimize flood risk with minimal investment, minimize loss of productive farmland and maintain a fair distribution of costs for Dike Ring 1 & 2?

Problem Framing

A problem can be interpreted and communicated in many ways. According to De Bruijn (2019), a frame can be defined in two ways: as a filter through which people are perceiving the world and as the structure of a message aiming at a specific interpretation of the world. These definitions of framing can be interesting to use in the Flood Risk Management Plan in regard to the different problem perspectives of all the actors involved. The goal from this chapter is to see how the decision problem about the Flood Risk Management Plan can be structured in many ways, and how this influences the political arena where Dike Ring 1 & 2 operate.

There are supposed ten actors involved in the decision-making process of this plan with various perspectives on how they perceive the problem. Due to some reasons, groups representing Rijkwaterstaat were inactive and thus no Rijkwaterstaat actor during the decision-making rounds.

  • The Delta Commission
  • Gelderland Province
  • Overijssel Province
  • Dike Ring 1 & 2
  • Dike Ring 3
  • Dike Ring 4
  • Dike Ring 5
  • Environmental Activist
  • Transport Company
  • Rijkwaterstaat

The Flood Risk Management Plan (FRMP) of the IJssel River is a project commissioned by the Delta Commission. Their main goal is to minimize the risk of flooding of the upper branch of the Ijssel River without any strong preference where the Room for the River Project will take place.

Provinces of Gelderland and Overijssel want to increase their safety too. However, the Room for the River Project often widen the river and thus reducing productive land. Both of them prefer to place it in the other province.

Four actors representing five different dike rings have a similar perspective. Every dike ring has its own reason to protect their land. There are economic value and historical building that want to be protected.

The environmental activist is a special actor that really push the Room for the River Project. They perceived the project will revitalize the ecosystem of the river. On the other hand, the transport company oppose the Room for the Project itself as it will reduce their ability to utilize the river as their mean of transporting logistics. If the environmental activist argues that it’s good for nature due to its revitalization, the transport company argue that it’s bad because of increasing carbon emission.

As Dike Ring 1 & 2 Analyst, our report will give advice specifically for Dike Ring 1 &2 with their preference in mind. Having said that, our client opposed to having the Room for the River implemented on their part because it means they need to sacrifice some of their modern style farmlands.


A model is used to explore the most effective ways to utilize mitigation measures. XLRM framework is drawn to explain the model.

Figure 2: IJssel River Flood XLRM Model

We intentionally focused on Dike Ring 1 & 2 Outcomes and sum up the other Dike Ring Performance Metric. The Room for the River Project was charged separately despite the implementation location. It benefits further than the Dike Ring itself. A breach in one of the Dike Ring will also evacuate all people in the province instead of in the local area.

We run the model to simulate 60 years divided into three periods of 20 years. Facing several uncertainties with unpredictable model behaviour, the model was run a hundred thousand times then all of the outcomes were collected. Multi scenario MORDM was chosen to provide advice for our client. Its method could be afforded by our computational power and relatively find robust policies (compared to MORDM). EMA Workbench package in Python was used extensively for this project.

Figure 3: Multi-scenario MORDM Flowchart

To start off the first round of optimization, a reference scenario was needed. We chose values halfway through the range in the hope that the scenario would as neutral as possible.

To optimize under a scenario to find a set of candidate solutions a multi-objective evolutionary algorithm (MOEA) was used. According to Kasprzyk, Reed and Hadka, an MOEA can find innovative diverse solutions (2016). It is a population-based search technique that uses a process based on natural selection to evolve solutions which have a good performance and in this way find the optimal solution.

The use of an MOEA is relevant to the optimization as it can take into account uncertainties (Maier et al., 2019) and it is able to handle multi-objective (Deb, 2015).

The MOEA uses the ɛ-NSGA-II multi-objective evolutionary algorithm. It is one of the most applied algorithms (Garcia & Trinh, 2019) and provides an accessible approach to using an MOEA. We decided to use epsilon progress and hypervolume as our convergence metrics. Convergence metrics are used to analyze whether the solution that was found achieved near the optimum point.

To have good convergence, a sufficient amount of function evaluations have to be considered. 20000 number of function evaluations (nfe) was run for the first round and 10000 nfe for the second round of optimization for three worst-performing scenarios.



The metrics used to identify whether the optimization algorithm has converged were the hypervolume and the epsilon progress.

Figure 4: Convergence Metrics of optimization of the first phase reference scenario

In the first round of optimization under one reference scenario, no convergence was established. The fact that the optimization algorithm has not converged increases the likelihood of not having found an optimal solution. Although this was already the case during the first optimization, we choose to continue the multi-scenario MORDM process nonetheless.

Figure 5: Convergence measurement of second phase multi scenario optimization

The hypervolume indicator in scenario 2 the value seems to stabilize. However, the epsilon progress does not stabilize yet. Scenario 2 did not converge. Scenario 0 on the other hand clearly still has increasing epsilon progress and the hypervolume there plummets after about 8000 nfe. Reason for this could be that the maximum of the hypervolume was chosen incorrectly and therefore it suddenly dropped. From about 2000 nfe for scenario 1 the hypervolume only slightly increases and then plummeted. It potentially happened due to incorrect hypervolume boundary since the hypervolume plummets whereas the epsilon progress seems to stabilize. In the optimization under multiple scenarios, convergence was also not established.

Multi-Scenario MORDM Results

The first run for reference scenario – 20000 nfe (number of function evaluation) found 108 solutions. We selected 7 solutions with the lowest expected number of deaths. 7 solution candidate that gives us the lowest number of death then inputted into the model with 1000 scenario variations. We evaluate them based on the maximum regret range.

We choose the three scenarios that giving the worst outcomes and then set it as a new reference set of scenarios. Each of them runs 10000 nfe and found 22 solutions, 1 solution and 24 solutions respectively.

Figure 6: Parcoord of policy optimization for the three worst-performing scenarios

We select again with the lowest expected number of deaths with the same criteria leaving us with 11 solution candidates. Continued to run 1000 scenario evaluations. We evaluate them again based on the maximum regret range.

Figure 7: Visualization of robustness performance of selected policies

Most Robust Policies: To find a solution that is as robust as possible, we need to look for the minimum values on the maximum regret axes that are shown in the parcoord visualization. Policies that can be identified as being robust under the scenarios that were covered amongst others are policy 7 and policy 11. Policy 7 is the limited RfR measures policy. RfR measures are implemented in Dike Ring 3 and 4. In policy 11, RfR measures are implemented in all Dike Rings in between the sixty years period.

Sensitivity Analysis

A global sensitivity analysis was performed to look deeper at the dike model vulnerabilities. A heatmap was made to show how dependent the outcomes affected by the uncertainties.

Figure 8: Sensitivity Analysis Heatmap

Variance in outcomes of expected annual damage and the expected number of deaths in A.1 is mostly dependent on the A.1_pfail. The variance of expected annual damage and the expected number of deaths in A.2 is mostly dependent on the A.2_fail. The variance expected annual damage and the expected number of deaths at dike ring 3, 4 and 5 is mostly dependent on A.3_pfail and A.5_pfail. As can be expected, the variance in the expected evacuation costs is mostly dependent on the probability that a dike fails. It is however interesting to note that this is mostly dependent on the A.2_pfail, A.3_pfail and A.5_pfail.

The policies found with the dike model are thus vulnerable to the probability that the dike fails at A.1, A.2, A.3 or A.5. It is interesting to see A.4 fail probability affect the policy lightly. Further analysis could be done to explain the phenomenon.



  1. There is no guarantee that we found the optimal robust solution as there is always a trade-off between robustness and optimality.
  2. The reference scenario is chosen arbitrarily. Although it’s considered neutral, we do not have the expertise to guarantee it.
  3. The first and second optimization have not reached stable convergence metrics.


Following the results and discussion, several recommendations could be made in order to facilitate future research. One of the crucial ways to improve the analysis is to reach more convergence. More model runs could significantly improve the working of the MORDM approach. Spending more time and computational power will result in a better iterative process of scenario discovery and refining policies (Kwakkel, Walker, & Haasnoot, 2016).

In this analysis, PRIM can be used as a base for reference scenarios. This provides a more solid base in looking into the reference scenario than opting for a scenario based on halfway through the possible range as this might be way less likely than in reality. Choosing the reference is crucial and the first step. The benefit of this is that the worst scenarios are ruled and this has the potential to include better scenarios.

Another recommendation is to involve the multi-actor perspectives more intensively. This decision arena is full of actors with different interests and levels of power. Modelling with stakeholders could lead to more negotiated knowledge and could be based on overlapping consensus. However, it is worth to note the participative modelling approach can become very time consuming and labour-intensive. This approach is more likely to succeed if values are aligned, which is the case looking at the actor analysis and the problem framing.

An alternative approach to these kinds of problems with multiple actors is to act as a policy broker. This means that the modeller shifts from how to get the model accepted to how to become attractive for the different actors involved. Especially when looking at the different dike rings and government bodies, it would be helpful to identify win-wins in future research. The research could be aimed at answering what-if questions and in exploring more combination of possible policies.


This study has been executed for Dike Ring 1&2 to explore possible policies regarding the new flood risk management plan. There are different levers to be pulled in order to ensure the safety of the citizens of Dike Ring 1&2. It is possible to avoid Room for the River Project implemented in Dike Ring 1 & 2 Region. Solutions that we found at least hinted so. The following problem statement was important in finding robust policies: how to minimize flood risk with minimal investment, loss of productive farmland and a fair distribution of costs?

The MORDM approach is used to discover three scenarios in order to find different policies. Ultimately eleven policies remained after an iterative process of optimization. Sensitivity analysis and robustness metrics on these policies lead to two policies that are considered as most robust. Policy 7 and policy 11 are the most interesting policies for Dike Ring 1&2 to take into consideration. These policies come with minimal investment and minimal loss of farmland.

Policy 7 and policy 11 contain Room for the River measures implemented at different dike rings at different starting time. It is done in order to spread the cost, so no dike ring should have losses most of their land. These policies also signify dike heightening is needed in different timesteps. The overlap between those policies is that room for the river measures would be implemented at both dike ring 3 and dike ring 4.


  • Bartholomew, E., & Kwakkel, J. H. (2020). On considering robustness in the search phase of Robust Decision Making: A comparison of Many-Objective Robust Decision Making, multi-scenario Many-Objective Robust Decision Making, and Many Objective Robust Optimization. Environmental Modelling and Software, 127(March), 104699.
  • van der Brugge, R., Rotmans, J. & Loorbach, D. (2005). The transition in Dutch water management. Reg Environ Change 5, 164–176.
  • de Bruijn, H. (2019). The Art of Political Framing: How Politicians Convince Us that They are Right. Amsterdam University Press.
  • Bryant, B. P., & Lempert, R. J. (2010). Thinking inside the box: A participatory, computer-assisted approach to scenario discovery. Technological Forecasting and Social Change, 77(1), 34–49.
  • Cariboni, J., Gatelli, D., Liska, R., & Saltelli, A. (2007). The role of sensitivity analysis in ecological modelling. Ecological Modelling, 203(1–2), 167–182.
  • Ciullo, A. (n.d.). Case study presentation [Powerpoints]. Retrieved from
  • Deb, K. (2015). Multi-Objective Evolutionary Algorithms. In J. Kacprzyk & W. Pedrycz (Eds.), Springer Handbook of Computational Intelligence (pp. 995–1015). Springer.
  • Garcia, S., & Trinh, C. T. (2019). Comparison of multi-objective evolutionary algorithms to solve the modular cell design problem for novel biocatalysis. Processes, 7(6).
  • van Houdt, J. (October 9, 2009). NL 09 10 09 luchtfoto Ruimte voor de rivier IJssel Arnhem Kampen ID366111 [Photo].
  • Jaxa-Rozen, M., & Kwakkel, J. (2018). Tree-based ensemble methods for sensitivity analysis of environmental models: A performance comparison with Sobol and Morris techniques. Environmental Modelling and Software, 107(June), 245–266.
  • Kasprzyk, J. R., Nataraj, S., Reed, P. M., & Lempert, R. J. (2013). Many objective robust decision making for complex environmental systems undergoing change. Environmental Modelling and Software, 42(1), 55–71.
  • Kasprzyk, J. R., Reed, P. M., & Hadka, D. M. (2016). Battling Arrow’s Paradox to discover robust water management alternatives. Journal of Water Resources Planning and Management, 142(2), 1–12.
  • Kwakkel, J. H., & Haasnoot, M. (2019). Supporting DMDU: A Taxonomy of Approaches and Tools. In V. A. W. J. Marchau, W. E. Walker, P. J. T. M. Bloemen, & S. W. Popper (Eds.), Decision Making under Deep Uncertainty: From Theory to Practice. Springer International Publishing.
  • Maier, H. R., Razavi, S., Kapelan, Z., Matott, L. S., Kasprzyk, J., & Tolson, B. A. (2019). Introductory overview: Optimization using evolutionary algorithms and other metaheuristics. Environmental Modelling and Software, 114(June 2018), 195–213.
  • McPhail, C., Maier, H. R., Kwakkel, J. H., Giuliani, M., Castelletti, A., & Westra, S. (2018). Robustness Metrics: How Are They Calculated, When Should They Be Used and Why Do They Give Different Results? Earth’s Future, 6(2), 169–191. ttps://
  • Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., & Wu, Q. (2019). Why so many published sensitivity analyses are false: A systematic review of sensitivity analysis practices. Environmental Modelling and Software, 114(March 2018), 29–39.
  • van Urk, G. (1978.) The macrobenthos of the river IJssel. Hydrobiological Bulletin 12(1), 21–29.
  • Venterink, H.O., Wiegman, F., van der Lee, G.E.M. & Vermaat, J.E. (2003). Role of Active Floodplains for Nutrient Retention in the River Rhine. Journal of Environmental Quality, 42(4), 1430-1435.
  • Walker, W. E., Marchau, V. A. W. J., & Kwakkel, J. H. (2013). Uncertainty in the Framework of Policy Analysis. In W. W. Thissen W. (Ed.), Public Policy Analysis. International Series in Operations Research & Management Science (Vol. 179, pp. 215–261). Springer.
  • Waterman, R.E., Misdorp, R. & Mol, A. (1998). Interactions between water and land in The Netherlands. J Coast Conserv, 4(1), 115–126.