Composite indicators have been increasingly recognized as a useful tool for performance monitoring, benchmarking comparisons and public communication in a wide range of fields. The usefulness of a composite indicator depends heavily on the underlying data aggregation scheme where multiple criteria decision analysis (MCDA) is commonly used. A problem in this application is the determination of an appropriate MCDA aggregation method. Of the many criteria for comparing MCDA methods, the Shannon-Spearman measure (SSM) is one that compares alternative MCDA aggregation methods in constructing composite indicators based on the information loss concept. This paper assesses the effectiveness of the SSM using Monte Carlo approach-based uncertain analysis and variance-based sensitivity analysis techniques. It is found that most of the variation in the SSM arises from the uncertainty in choosing an aggregation method. Therefore, the SSM can be considered as an effective measure for comparing MCDA aggregation methods in constructing composite indicators. We also use the SSM to evaluate five MCDA aggregation methods in constructing composite indicators and present the findings.