Abstract In the literature, there is a lack of tools able to optimize contextually the design and operation of a multi-energy system in its entirety, encompassing both (i) the number, type, and size of the energy conversion and storage plants supplying the end users of the system with the required energy and (ii) the geometry and capacity of the distribution networks delivering that energy to the users. Moreover, rarely the retrofit design problem is considered, where “retrofit design” refers to the addition of new capacity to components initially available in existing systems. Here, a general method is proposed to simultaneously optimize the retrofit design and operation of a multi-energy system and the associated energy networks. The goal consists of finding the additional capacity to be added to the already available components—energy conversion and storage plants, energy networks—and the new components to be installed in order to comply with given reduction targets in carbon emissions while keeping the life cycle cost of the system at a minimum. A district composed of commercial and residential buildings operating in a microgrid is considered as a case study. Heat can be provided to the end users via a district heating network, while electricity can be either generated on-site or imported from the national power grid. Results of the retrofit design problem show a contextual reduction of 35% in CO2 emission and 20% in life cycle cost with respect to the original system configuration.