Geothermal heat pumps have a widespread diffusion as they are able to deliver relatively higher energy output than other systems for building air-conditioning. The exploitation of low-enthalpy geothermal energy, however, presents crucial sustainability issues. This review investigates the primary forms of the environmental impact of geothermal heat pumps and the strategies for their mitigation. As life-cycle analyses shows that the highest impacts arise from installation and operation stages, most optimization studies focus on system thermodynamics, aiming at maximizing the energy performance via the optimization in the design of the different components interacting with the ground and serviced building. There are environmental studies of great relevance that investigate how the climate and ground properties affect the system sustainability and map the most suitable location for geothermal exploitation. Based on this review, ground-source heat pumps are a promising technology for the decarbonization of the building sector. However, a sustainable design of such systems is more complex than conventional air-conditioning systems, and it needs a holistic and multi-disciplinary approach to include the broad environmental boundaries to fully understand the environmental consequences of their operation.