Extreme conditions caused by climate change and other crises call for enhancing the resilience of buildings and urban energy systems. This paper investigates the role of collaborative decision-making to improve the performance of single buildings and the unified whole in the form of a cohesive cluster of energy consumers to enhance resilience. CIRLEM, the previously developed energy management approach, provides flexibility in energy systems through collective behavior of entities and deploying a lightweight Reinforcement Learning algorithm. This research contributes to developing a novel signal generation structure including price- and demand-based function to stimulate the cohesion attribute. Extended thermal comfort margins are introduced to broaden the flexibility potential, and reward function includes thermal zones categories. The energy management approach and extended comfort constraints is tested under an extreme cold winter in a pilot ecosystem located in Norway made of several buildings characterized by different sizes, use types, performance and energy systems. Acting individually, buildings could save 28 % and 13 % energy and cost, while acting as a collaborative cluster, energy use and cost are reduced by 42 % and 40 %. Through collaboration between buildings, high-performance buildings could help others under high energy demand periods to keep their functionality toward the cluster’s goal.