Climate change is expected to severely affect cropping systems and food production in many parts of the world unless local adaptation can ameliorate these impacts. Ensembles of crop simulation models can be useful tools for assessing if proposed adaptation options are capable of achieving target yields, whilst also quantifying the share of uncertainty in the simulated crop impact resulting from the crop models themselves. Although some studies have analysed the influence of ensemble size on model outcomes, the effect of ensemble composition has not yet been properly appraised. Moreover, results and derived recommendations typically rely on averaged ensemble simulation results without accounting sufficiently for the spread of model outcomes. Therefore, we developed an Ensemble Outcome Agreement (EOA) index, which analyses the effect of changes in composition and size of a multi-model ensemble (MME) to evaluate the level of agreement between MME outcomes with respect to a given hypothesis (e.g. that adaptation measures result in positive crop responses). We analysed the recommendations of a previous study performed with an ensemble of 17 crop models and testing 54 adaptation options for rainfed winter wheat (Triticum aestivum L.) at Lleida (NE Spain) under perturbed conditions of temperature, precipitation and atmospheric CO2 concentration. Our results confirmed that most adaptations recommended in the previous study have a positive effect. However, we also showed that some options did not remain recommendable in specific conditions if different ensembles were considered. Using EOA, we were able to identify the adaptation options for which there is high confidence in their effectiveness at enhancing yields, even under severe climate perturbations. These include substituting spring wheat for winter wheat combined with earlier sowing dates and standard or longer duration cultivars, or introducing supplementary irrigation, the latter increasing EOA values in all cases. There is low confidence in recovering yields to baseline levels, although this target could be attained for some adaptation options under moderate climate perturbations. Recommendations derived from such robust results may provide crucial information for stakeholders seeking to implement adaptation measures.