Validity Of Scoring Methods In The Presence Of Outliers

Abstract

Rankings enjoy growing popularity in the economical sciences. Well known institutions like the World Economic Forum, Heritage Foundation and the OECD make use of rankings to exert competitive pressure on the ranked countries. To achieve any such desired effects rankings need to be accepted and approved as a whole, and in particular regarding the applied methodology. In order to appeal to wide sections of the population scoring methods are applied to aggregate a composite indicator. Experience has shown that outliers have a distorting effect on the ranking order and therefore cause economically implausible results which are a target for criticism. For these reasons the choice of an adequate scoring method is of great importance. The applied technique should provide a feature which enables it to mitigate the distorting effect of outliers without the necessity for an arbitrarily elimination of data points. Although scoring methods have a high influence on ranking results, scientific analysis is often more concerned with the optimal choice of indicators or the weighting scheme, whereas the impact of extreme values is not addressed. According to this, the present research is related to the question which scoring method is the best choice in the presence of outliers. Evidence is given, that Logistic Function Methods have the ability to mitigate outlier distortion effects. The analysis approaches the issue considering two aspects: It combines the theoretically derivation of scoring methods’ statistic strengths and weaknesses for the ranking process and highlights the bootstrap technique to assess the validity of score results in the presence of outliers.