Abstract: Various Collective Heating and Cooling Systems (CHCS) have emerged as promising low-carbon energy solutions for buildings. However, the absence of tailored decision guidelines often hinders decision-makers from identifying the optimal system for any given case. This research introduces a novel methodology for comprehensive evaluation of different CHCS under diverse case-specific boundary conditions, leading to informed recommendations. The proposed methodology integrates occupants' preferences for thermal comfort and costs into a holistic Key Performance Indicator (KPI) score, i.e. a weighted sum of normalised indicators including indoor thermal comfort, domestic hot water comfort, and levelised cost of energy. By applying this methodology to evaluate three advanced central change-over temperature CHCS across various building sizes and family types, our study demonstrates the effectiveness of this approach. The results suggest that 4-pipe systems are preferable when prioritising thermal comfort, whereas decentralised booster heat pumps are recommended for cost reduction. Notably, for small apartment buildings inhabited by working families, a 2-pipe system with decentralised storage might be preferred. These insights underscore the importance of incorporating occupants' preferences into multi-objective decision-making. Furthermore, the holistic KPI score methodology can assess different control strategies and provide valuable insights for policymakers when extended with additional indicators.