Policy makers dealing with composite indicators are often confronted with the issues of the robustness of composite indicators and of the reliability of the concluded policy messages. In the Communication from the Commission COM (2000) 619 one can read: “[…because composite indicators invite strong policy messages to be concluded, they need to be robust and based on a sound methodology]”. This final report of the Exploratory Research, “Integration of thematic composite indicators” provides first a review of both methodological aspects and practical test cases of composite indicators. Several methods are investigated such as aggregation systems, multiple linear regression models, principal components analysis and factor analysis, cronbach alpha, neutralization of correlation effect, efficiency frontier, distance to targets, experts opinion (budget allocation), public opinion, and Analytic Hierarchy Process. The report further presents concisely twenty-four published studies on composite indicators in a number of fields such as environment, economy, research, technology and health, including practices from the Directorates General of the European Commission. For each composite indicator reviewed, general information is provided, on the number and type of sub-indicators, on the preliminary treatment (normalisation, detrending etc.) and on the weighting system considered. Finally, a brief assessment of each composite indicator regarding the appropriateness of the selected aggregation methodology, in terms of the complexity and inter-correlations of the data, is offered. This ‘state-of-the-art’ aims at creating a background in the development of composite indicators and at identifying the main problems encountered when constructing composite indicators. One of the most common problems is the plurality of perspectives on the relative importance of the sub-indicators (variables) and its influence on the outcomes of the composite indicator. To this end, a number of statistical tests have been framed. We propose the use of uncertainty analysis (UA) and sensitivity analysis (SA) to gain useful insights during the process of composite indicators building, as well as in the assessment of their quality, their transparency and their suitability to defend policy messages. These issues are discussed on the Technology Achievement Index (TAI), a composite indicator developed by the United Nations (2001). Two participatory methods have been used for assigning weights to the sub-indicators (Analytical Hierarchy Process and Budget Allocation), following a pilot survey among JRC informed individuals. The statistical treatment for robustness analysis of the composite indicators has been deemed necessary by several Directorates General of the Commission, including: MARKET, ECFIN, RTD, EMPL and ENTR.
Year of publication | |
Authors | |
Geographic coverage | Others |
Originally published | 15 Nov 2018 |
Knowledge service | Metadata | Composite Indicators |