Skip to main content

Supporting policy with scientific evidence

We mobilise people and resources to create, curate, make sense of and use knowledge to inform policymaking across Europe.

Publication | 4 June 2020

JRC Statistical Audit of commitment to reducing inequality index 2017

Composite Indicators

Conceptual and practical challenges are inevitable when trying to summarise with a single composite indicator the commitment of countries to reducing inequality. The summary and section 5 of the Commitment to Reducing Inequality Index report discusses in detail the conceptual framework, while the selection of indicators, data quality aspects and methodological choices for grouping country-level data across 8 main indicators, 3 pillars and an overall index are presented in the Methodology Annex.

The statistical audit presented below constitutes the first collaboration between Oxfam and the European Commission’s Joint Research Centre (JRC), specifically the Competence Centre on Composite Indicators and Scoreboards. The statistical assessment carried out by JRC aims to contribute to ensuring the transparency and reliability of the Commitment to Reducing Inequality (CRI) index and thus to enable policymakers to derive more accurate and meaningful conclusions, and to potentially guide choices on priority setting and policy formulation.

Statistical soundness should be regarded as a necessary but not a sufficient condition for a sound index, since the correlations underpinning the majority of the statistical analyses carried out herein “need not necessarily represent the real influence of the individual indicators on the phenomenon being measured”.1 The development of any index must thus be nurtured by a dynamic iterative dialogue between the principles of statistical and conceptual soundness. In that respect, prior to undertaking the present statistical assessment, Oxfam and JRC engaged in previous discussions during spring 2017. An earlier version of the CRI index was assessed by the JRC in March–April 2017. Fine-tuning suggestions, aimed at setting the foundation for a balanced index, were taken into account by Oxfam and Development Finance International research teams for the final computation of the CRI scores and rankings.

The JRC assessment of the CRI index presented in this appendix has focused on two main issues: the statistical coherence of the structure, and the impact of key modelling assumptions on the CRI scores and ranks.2 In particular, the JRC analysis complements the reported country rankings for the CRI index with estimated confidence intervals, in order to better appreciate the robustness of these ranks to some modelling choices (such as the weighting scheme, aggregation formula and estimation of missing values).

Overall, the main conclusions of the present audit can be summarised as follows: the CRI Index is representative of a plurality of scenarios, reliable and with a statistically coherent framework. The uncertainty analysis shows that country ranks are robust for one-third of the countries. For a number of countries, in particular non-OECD countries, ranks should be analysed within their expected confidence intervals instead of being taken at face value. The statistical assessment has also shown that the CRI index has a good statistical reliability and measures one single latent phenomenon capturing the three main components of the index, the Spending, Tax and Labour pillars. Notwithstanding the good statistical properties of the CRI index, some suggestions are made for possible refinements of the CRI index in future editions once more data and time series are available.