"Preference Modelling as an activity for structuring valid data during MCDA intervention" and some further considerations (A reply to Anna Ostanello)
Denis Bouyssou, Patrice Perny, Marc Pirlot, Alexis Tsoukiàs, Philippe Vincke
In the Spring 96 number of the Newsletter of the EURO working group on MCDA Anna Ostanello presented a very stimulating contribution on which we mostly agree. In particular, we share the view that the study and conceptualization of decision aid in organizational contexts is a major issue, probably the most important one, for today's MCDA. We also agree with her suggestion for opening the traditional framework of MCDA and studying relevant aspects of human interaction (or man-machine interaction). We disagree however on some points related to her presentation of the current state of the field. Anna criticizes the narrow-mindedness of focusing on the situation of an ideal isolated DM ("hypothetical decision maker freed from any context or process constraints"). That's all right! We agree in some sense when she contests "mathematicians" to have exerted a control on the development of the field, the aim being to "rationalize the decision process" like within utility theory, though in a weaker form. But the way this is stated is misleading! It sounds as if the mass of the production in MCDA had been theorems, axioms and the like whereas the main achievement in the orbit of the EURO Working Group has been the setting up of methodologies for decision aid (and, more often, of methods without methodologies). With very few yet notable exceptions, this field of research has not produced so much theory or mathematical results. In particular, there is a cruel lack of a comprehensive framework which would facilitate the understanding of the differences and resemblances between procedures and methods that have been proposed; basic concepts such as compensation, outranking, importance of criteria, intensity of preference, ...have not aroused that much theoretical investigation. So, we don't think that the focus has been on theory. By no means. We believe that even the ideal case of the isolated DM is far from having been fully investigated.
So, we advocate, as one of the openings of the framework, suggested by Anna, a (re)new(ed) interest for theoretical investigation. In the following we will try to explain further our views in relation with Anna's while introducing also some more general ideas about MCDA.
MCDA (and other OR areas) is a constrained human activity. Constrained by many factors, and in particular the languages that are used among which, often, some mathematics or formal models. It is true that such a language may be very strong towards real problem situations. But it is also true that a specific feature of this area is its ÒformalÓ approach, where by ÒformalÓ we mean a language where ambiguity is reduced as much as possible (human languages are not formal although perhaps more powerful). It has clearly appeared that mathematical models are not appropriate in all decision situations, for instance those at a strategic level where it is hardly possible to account in unambiguous mathematical terms for complex interactions between actors, constraints imposed by the environment, etc. In these cases where systemic aspects are prevailing, conceptualization efforts have been made resulting in methodologies and tools for structuring problems in an interaction process with the actors; as Anna says, further efforts are needed to better integrate the multicriteria aspects. So, it is certainly possible to provide valid and satisfying decision aid without using any formal approach and by no means it should be considered weak or irrelevant (although, unfortunately it happens to be so in some studies).
However, in some cases, computations are made, operations are performed on mathematical structures ( e.g. a weighted sum is computed) and conclusions in natural language are derived from the mathematical objects obtained through these operations. Underlying models are not always (not often) made explicit. What we believe is that there should be as strong a coherence as possible between the formulation of the problem in natural language (or more generally what emerges from the interactions of the actors of the decision aid process) and (at least) one of the formal models compatible with the manner in which the mathematical operations are performed. To give an example, numbers which reflect ordinal properties of alternatives should not, in principle, be used in a weighted sum. Using ordinal information (i.e. information which appears as ordinal from the interaction process) in such an operation is simply unfair; it could be interpreted as trying to "influence" the process "; alternatively, it is much like a wrong translation from a language into another; if this results in substantial perturbation of the original meaning of the sentence and if you start elaborating on the basis of this misinterpretation, you might be led to derive conclusions that are inconsistent with the original statement. Stating it in still another manner, it is similar to reasoning by analogy: if the analogy is not deep enough, the conclusions might be irrelevant. We think that such a semantic coherence or faithfulness between formulations of problems in different languages (be they mathematics or other formal or natural languages, be they from a technical to a non-technical language) is an essential feature for the validation of a decision aid process. The "Analyst" should not only be a facilitator, he should also be a "chartered translator". We hear an echo of this in Anna's considerations about validity for instance when she writes "... (external validity) ... might not be sufficient to guarantee the adequacy of these 'data' to be an input for a subsequent tool application for a formal modelling. The analyst has, therefore, to carry out new 'internal' validation tests on such input". It is precisely the role of mathematical or axiomatic analysis to describe the conditions on such "inputs" which will allow to faithfully interpret them as particular formal objects (numbers, relations,...); the analysis should also suggest which operations can be meaningfully performed on such objects in view of their semantic content. So, we believe that theoretical investigation should be encouraged in this perspective, as a background for helping the analyst to provide valid decision aid. Recall that in our conception, axioms are not telling you how to behave if you want to be considered "rational", they are not norms of rational behaviour; they are descriptions of procedures, descriptions which hopefully help you to identify the procedures or operations that could be used in a given context. Mathematical investigation should be a process where theory and practice interact and question each other.
So it is clear for us that we have to "open our cultural framework", but "without throwing the dirty water and the baby". Moreover we claim that such an opening should consolidate and validate theoretical results concerning MCDA. After all in order to make an excellent wine we need both a "sommelier" and a chemical engineer. The former has an "holistic" and informal point of view, while the later has a "reductionist" and formal point of view. We claim we need the both. On the one hand we need to improve our "formal languages" (as happens for instance when we introduce non classical logic as meta-mathematics of preference modelling in order to treat ambiguity in a non ambiguous way). On the other hand we need to improve validation and understanding of decision aid processes (as Anna herself presents in her contribution).
The complementarity of the two approaches can also be observed as far as the legitimation of MCDA is concerned. Legitimation has to be both operational (ability to provide satisfactory and meaningful results) and theoretical (correctness and ability to innovate). Operational legitimation could be measured by the penetration of MCDA in the "decision aid domain", establishing reference concepts and standard procedures. Theoretical legitimation could be measured by the attractiveness of MCDA for young researchers and for other research areas and by the presentation of outstanding theoretical results.
The two legitimation are not independent and cannot be even be conceived as such, due to the specific status of MCDA, which is not a mathematical theory, but an open theoretical field on the borderline between "science" and "craft". Validation, axiomatization and conceptualization should improve together and in synergy.
Finally we want to present a question on which a lot of confusion is done while discussing future of MCDA (and perhaps OR more generally). Researchers are not practitioners (and viceversa). This simple distinction (although not necessarily so crisp) states that many questions concerning "MCDA applications", "real case studies" etc., are typical problems of young theories on which it is not yet available solid experience on how to use them. What we claim is that researchers should provide practitioners with sufficiently clear tools and knowledge so that it is possible to cover the gap (besides learning from practitioners). But how practitioners will effectively use such tools in real decision situations cannot be anticipated. This is not to mean that researchers should not take care of how theory is used, but that is out of scope the simulation of decision aid practice. Researchers provide knowledge, practitioners take their chances!
Last, but not least, the transmission of knowledge in our area has to be considered as a critical issue. Multicriteria analysis is still considered as a special case of conventional operational research and is teached as such, if ever. To our knowledge specific MCDA classes in Universities are not frequent. Moreover it is also rare to find consolidated professional training in the use and application of MCDA. We claim that the maturity of a scientific domain is solid if knowledge transmission is such.
We are really glad that in the last few years the discussion concerning the above issues and the future of the MCDA increase and improve. We hope that this process will continue and help to open fruitful new routes.