**David Ríos Insua
**

*Decision Engineering Lab,
*

*U. Rey Juan Carlos-DMR Consulting Foundation
*

*Spain
*

**Introduction
**

The latest issues of the
EWG-MCDA Newsletter have presented several views on what is robustness analysis.
In this note, I would like to complement them describing what is understood by
such term, within the Bayesian arena. Further details may be seen in Ruggeri et
al (2004). The Bayesian approach to inference and decision analysis, see French
and Rios Insua (2000), essentially suggests:

- Modelling beliefs about a parameter of interest through a prior which, in presence of further information, is updated to the posterior.
- Modelling preferences and risk attitudes about (multicriteria) consequences, with a multiattribute utility function.

- Associate with each alternative its (multiattribute) posterior expected utility.
- Propose the alternative which maximises the posterior expected utility.

As in any quantitative
approach, there are many reasons to check the sensitivity of the output (the
optimal alternative) with respect to the inputs (model, beliefs and
preferences). In addition, since, in this framework, inputs to the analysis
encode the DM's judgements, she should wish to explore their implications and
possible inconsistencies. The need for sensitivity analysis is further
emphasised by the fact that the assessment of beliefs and preferences is a
difficult task. This is an especially important point, as her judgements will
evolve through the analysis until they are *requisite*.
Robust Bayesian analysis guides this process.

The usual
practical motivation underlying robust Bayesian analysis is the difficulty in
assessing the prior distribution. Consider the simplest case in which it is
desired to elicit a prior over a finite set of states *Θi, i=1,…,I*. A common technique to assess a precise *Π(Θi)=
pi*, with the aid of a reference experiment, proceeds as follows: one
progressively bounds *Π(Θi)* above and below until no further discrimination is
possible and then takes the midpoint of the resulting interval as the value of *pi*.
Instead, however, one could directly operate with the obtained constraints *αi
<= Π(Θi) <= βi*,
acknowledging the cognitive limitations.

The same situation holds when modelling preferences. One might assess the utility of some consequences through, say, the certainty equivalent method, and then fit a utility function. However, in reality, we only end up with upper and lower constraints on such utilities, possibly with qualitative features such as monotonicity and concavity, if preferences are increasing and risk averse. These constraints can often be approximated by an upper and a lower utility function, leading to the consideration of all utility functions that lie between these bounds. If a parametrised utility function is assessed, the constraints are typically placed on the parameters of the utility, say the risk aversion coefficient. Of course, in developing the model for the data itself there is typically great imprecision, and a need for careful study of model robustness.

A final comment concerning the limits of elicitation concerns the situation in which there are several decision makers and/or experts involved in the elicitation. Then it is not even necessarily possible theoretically to obtain a single model, prior, or utility; one might be left with only classes of each, corresponding to differing expert opinions.

**Basic concepts
**

Robust Bayesian analysis provides tools to check the impact of the utility function, the prior and the model on the optimal alternative, and its posterior expected utility. We distinguish three main approaches to Bayesian robustness. We illustrate it considering robustness with respect to changes in the prior, but similar issues are raised when considering likelihoods and utilities. A “guided tour” through these three approaches is presented in Berger et al. (2000) and the references therein.

**Informal approach
**

The first approach is the *informal*
one, which considers several priors and compares the quantity of interest (e.g.,
the posterior mean) under them. The approach is very popular because of its
simplicity. Its rationale is that since we shall be dealing with messy
computational problems, why not analyse sensitivity by trying only some
alternative pairs of utilities and priors? While this is a healthy practice and
a good way to start a sensitivity analysis, in general this will not be enough
and we should undertake more formal analyses: the limited number of priors
chosen might not include some which are compatible with the prior knowledge and
could lead to very different values of the quantity.

It is worth mentioning that the consideration of a finite number of utilities links clearly with multi-objective decision making problems.

**Global robustness
**

The most popular approach in
Bayesian robustness is called *global
sensitivity*. All probability measures compatible with the prior knowledge
available are considered and robustness measures are computed as the prior
varies in a class. Computations are not always easy since they require the
evaluation of suprema and infima of quantities of interest.

The robustness measures provide, in general, a number that, in principle, should be interpreted in the following way:

- if the measure is “small”, then robustness is achieved and any prior in the class (possibly one computationally simple) can be chosen without relevant effects on the quantity of interest,

- if the measure is ``large'', then new data should be acquired and/or further elicitation narrows the class, recomputing the robustness measure and stopping as in the previous item; o.w. ….

- …. if the measure is “large” and the class
cannot be modified, then a prior can be chosen in the class but we should
be wary of the relevant influence of our choice on the quantity of
interest. Alternatively, we may use an ad hoc method such as
the
*G-minimax*, to select an alternative.

Given a class *G*
of prior measures, global sensitivity analysis will usually pay attention to the
range of variation of a posterior (or predictive) functional of interest
as the prior ranges over the class.

**Local robustness
**

The last approach looks for *local
sensitivity* and studies the rate of change in inferences and decisions,
using functional analysis differential techniques, such as Frechet derivatives
of posterior expected utilities and their norms, total derivatives or Gateaux
differentials.

**Decision and utility robustness
**

An important and occasionally controversial issue is the distinction between decision robustness and expected utility robustness. A variety of situations may hold. For instance, when performing sensitivity analysis, it may happen that expected utility changes considerably with virtually no change in the optimal Bayes action, even if the utility is fixed.

**Foundations
**

A number of results show that we
may model imprecision in beliefs and preferences through a class of probability distributions and a class of
utility functions. These results have two basic implications. First, they
provide a qualitative framework for sensitivity analysis, describing under what
conditions we may undertake the standard and natural sensitivity analysis
approach of perturbing the initial probability-utility assessments, within some
reasonable constraints. Second, they point to the basic solution concept of
robust approaches, thus indicating the basic computational objective in
sensitivity analysis, as long as we are interested in decision analytic
problems: that of *non-dominated
alternatives*. This corresponds to a Pareto ordering of decision rules, see
White (1982), based on inequalities on the posterior expected utility.

As a consequence of this model, the solution concept is the set of non-dominated alternatives. In some cases, non-dominance is a very powerful concept leading to a unique non-dominated alternative. However, in most cases the non-dominated set will be too large to imply a final decision. It may happen that there are several non-dominated alternatives and differences in expected utilities are non-negligible. If such is the case, we should look for additional information that would help us to reduce the classes, and, perhaps, reduce the non-dominated set. Some tools based on functional derivatives to elicit additional information may be seen in Martín and Ríos Insua (1997). Tools based on distance analysis may be seen in Ríos Insua (1990).

**Stability Theory
**

Stability theory provides
another unifying, general sensitivity framework, formalising the idea that
imprecisions in elicitation of beliefs and preferences should not affect the
optimal decision greatly. When *strong
stability* holds, careful enough elicitation leads to decisions with expected
utility close to the smallest achievable; when *weak
stability* holds, at least one stabilised decision will have such property.
However, when neither concept of stability applies, even small elicitation
errors may lead to disastrous results in terms of large losses in expected
utility.

**Conclusion
**

The approach we propose may be
summarised as follows: at a given stage of analysis, we elicit information on
the DM's beliefs and preferences, and consider the class of all priors and
utilities compatible with such information. We approximate the set of
non-dominated solutions; if these alternatives do not differ too much in
expected utility, we may stop the analysis; otherwise, we need to gather
additional information, possibly with the tools outlined above. This would
further constrain the class: in this case the set of non-dominated alternatives
will be smaller and we could hope that this iterative process would converge
until the non-dominated set is small enough to reach a final decision. It is
conceivable in this context that at some stage we might not be able to gather
additional information yet there remain several non-dominated alternatives with
very different expected utilities. In these situations, *L
× G-maximin solutions* may be useful as a way of selecting a single robust
solution. We associate with each alternative its worst expected utility; we then
suggest the alternative with maximum worst expected utility.

**Acknowledgements
**

Supported by grants from MCYT, URJC and DMR Consulting Foundation.

**References
**

1.
Berger, J.O., D. Rios Insua, and F. Ruggeri (2000). Bayesian robustness.
In D. Rios Insua and F. Ruggeri, eds., *Robust Bayesian Analysis*,
Springer-Verlag, New York, USA.

2.
French, S. and Rios Insua, D. (2000) *Statistical Decision Theory*,
Arnold.

3.
Martin, J. and Rios Insua, D. (1997) Local sensitivity analysis in
Bayesian Decision Making (with discussion) en Berger et al (eds) *Bayesian
Robustness*, 1997, Institute of Mathematical Statistics (con J. Martin),
119-135.

4.
Rios Insua, D. (1990). *Sensitivity Analysis in Multiobjective Decision
Making*. Springer-Verlag, New York, USA.

5.
Rios Insua, D., and F. Ruggeri (2000). *Robust Bayesian Analysis*.
Springer-Verlag, New York, USA.

6.
Ruggeri, F., Rios Insua, D. and J. Martin (2004) Robust Bayesian
Analysis, in Dey (ed) *Handbook of Statistics: Bayesian Analysis*, North
Holland. White, D.J. (1982). *Optimality and Eficiency*. Wiley, New York,
USA.

EWG-MCDA Newsletter, Series 3, No.9, Spring 2004

[HOME - EWG MCDA] [Newsletter articles of the EWG - MCDA]