About  Robustness Analysis


Philippe Vincke

SMG, Université Libre de Bruxelles

CP 210/01, 1050 Bruxelles, Belgium,



It is probably not useful any more to justify the introduction of the concept of robustness and to emphasise the interest of this concept in decision aiding. Confronted to the necessity (or, simply, the wish) to help a decision-maker, the analyst cannot avoid the presence of a lot of uncertainties, at least at three levels, as illustrated in figure 1. Traditional tools (like probability theory) or more recent ones (possibility theory, fuzzy sets, rough sets, ...) are useful but not sufficient to cope with all these uncertainties. Moreover they introduce themselves new uncertainties at the three levels of figure 1. So, we need a theoretical framework and methodologies to take into account the irreducible part of ignorance contained in any decision aiding process.


Definitions of robustness

No unique definition of robustness has been accepted by the scientific community until now and this is rather natural: the diversity of situations is so large that it will probably be necessary to classify the types of decision problems and the types of uncertainties before proposing different kinds of robustness which could be operational. In the literature, we can essentially distinguish 4 concepts, which could be starting points for future developments:

1.        the concept of robust decision in a dynamic context (Gupta and Rosenhead, 1972; Rosenhead et al., 1972; Rosenhead, 1989) which could also be called flexibility in the sense that a decision at a given time is robust if it keeps open as many ``good'' plans as possible for the future;

2.        the concept of robust solution in optimisation problems (Rosenblatt and Lee, 1987, in facility design problem; Mulvey et al., 1994, in mathematical programming; Kouvelis and Yu, 1997, in combinatorial optimisation) where robust means ``good in all or most versions'', a version being a plausible set of values for the data in the model;

3.        the concept of robust conclusion (Roy, 1998) where robust means ``valid in all or most versions'', a version being an acceptable set of values for the parameters of the model;

4.        the concept of robust method (Vincke, 1999 a, b, Sorensen, 2001) where robust means ``which gives results valid in all or most versions'', a version being a possible set of values for the data of the problem and for the parameters of the method.

Remark that we have adopted here the term “version”, recently proposed by B. Roy, instead of “scenario”, in order to avoid any reference to an unknown future and to the traditional probabilistic approaches.

There is no contradiction between these definitions: they only illustrate the fact that different kinds of robustness should be introduced in decision aiding. It is also important to avoid any confusion between robustness and the traditional stability property associated to sensitivity analysis. In this last context, a solution (decision) is determined in a particular version and an a posteriori study is made of the neighbourhood of that solution. The idea of robustness leads to consider, a priori, several versions (eventually rather different from each other) and to look for solutions (decisions, conclusions) which are good (valid) in all or most versions. In this perspective, the expression ``robustness analysis'' should be avoided because robustness considerations must be integrated during the decision aiding process and are not the result of an a posteriori analysis.


Robustness and MCDA

In the case where the decision problem is modelized as an optimisation problem and where a finite number of versions (sets of values for the data and the parameters of the model) has to be taken into account, one could argue that there are some similarities between searching for a good robust solution of the optimisation problem (that is a solution which is good in most versions and not too bad in the other) and searching for a good compromise solution of a multicriteria problem where the versions play the role of criteria. A concept like efficiency (non-dominance) could be used to select the candidates to the qualification of robust solutions and multicriteria methodologies could be applied to determine good robust solutions. The interested reader will find an illustration of that approach in Hites (2000), where the robustness of a solution does not only depend on its worst performance (as in Kouvelis and Yu) but simultaneously on its good an bad performances (without trivially applying an arithmetic or a weighted mean whose inconvenients were abundantly illustrated in Bouyssou et al., 2000). See also the concept of generalised Lorenz dominance used by Perny and Spanjaard (2002) in the same kind of problem.

Despite the similarities between searching for a good compromise solution of a multicriteria problem and searching for a good robust solution of a multiversions optimisation problem, one should avoid to consider that the only difference is the vocabulary (on this subject, see Hites et al., 2003). In the formulation of the problem, the family of criteria is built in such a way that the opinion of the decision-maker is as well represented as possible (cf. the concept of coherent family of criteria in Roy and Bouyssou, 1993), while the set of versions is often at least partially imposed by external conditions. Moreover the number of versions can be infinite (if the values of the parameters are defined through intervals) and the concepts of relative importance or preferential independence are not easy to transpose. Finally, most decision problems are simultaneously multicriteria and multiversions. In conclusion, we are convinced that the concept of robustness justifies the development of a specific theoretical framework and of new methodologies.


The subjective dimension of robustness

An important feature of robustness, in our mind, is its subjective dimension.

The fact that a decision (solution, conclusion) can be considered as robust depends on the more or less great margin the decision-maker is ready to concede in the information he wants to receive form the analyst. Let us consider an optimisation problem and suppose that the decision-maker is not affected by a difference of 5% between the values of different solutions. In this case, a solution whose value differs by less than 5% from the optimum in each version could be called robust (in the sense «good in all the versions »). Replacing 5% by another value will change the set of robust solutions. In another context, if you aggregate preferences in an outranking relation by using weights for the criteria, the robustness of the final relation (the versions being the sets of values for the weights) will depend on which modification of the relation is considered as negligible by the decision-maker. If he is very severe and considers that any modification is important, then imposing the robustness of the result will lead to a very poor relation (as it must be the same for all the sets of weights). But if he accepts some modifications (for example the replacement of some strict preferences by indifferences) then other robust results will be possible. More details on these examples and a proposition of theoretical framework in this direction were proposed in Vincke, 1999b.



Thirty years ago, the scientific community in decision aiding was confronted to the challenge of solving problems where several criteria were present. This led to the development of MCDA and to a lot of new concepts and tools. We are now facing to the challenge of taking into account the uncertainties, which are irremediably present in any decision aiding process. This probably justifies the development of a specific vocabulary, a specific theoretical framework, a new typology of the decision problems and new methodologies.


It is an open field for the future.



 N.B.: A list of references on robustness is maintained by Romina Hites at the following address then choice Research / Robustness. Every suggestion of new reference is welcome.


EWG-MCDA Newsletter, Series 3, No.8, Fall 2003

[HOME - EWG MCDA]    [Newsletter articles of the EWG - MCDA]