Abstract Building energy consumption is vulnerable to climate change due to the direct relationship between outside temperature and space cooling/heating. This work quantifies how the relationship between climate change and building energy consumption varies across a range of building types at different spatiotemporal scales based on estimates in 925 U.S. locations. Large increases in building energy consumption are found in the summer (e.g., 39% increase in August for the secondary school building), especially during the daytime (e.g., >100% increase for the warehouse building, 5–6 p.m.), while decreases are found in the winter. At the spatial scale of climate-zones, annual energy consumption changes range from −17% to +21%, while at the local scale, changes range from −20% to +24%. Buildings in the warm-humid (Southeast) climate zones show larger changes than those in other regions. The variation of impact within climate zones can be larger than the variation between climate zones, suggesting a potential bias when estimating climate-zone scale changes with a small number of representative locations. The large variations found in the relationship between climate change and building energy consumption highlight the importance of assessing climate change impacts at local scales, and the need for adaptation/mitigation strategies tailored to different building types.