Improving energy efficiency is essential for energy conservation, emissions reduction, and sustainable development. Prevalent huge efficiency gaps are not advantageous for the improvement of the region's overall energy efficiency. Although studies have analyzed the influencing factors of the regional energy efficiency gap, the impact of regional integration on the regional energy efficiency gap remains untested. This paper applies the extended stochastic frontier analysis (SFA) method that incorporates time-varying, time-invariant and city heterogeneous characteristics to estimate the city-level energy efficiency in China from 2005 to 2017. Building on the “center-periphery” framework, we further calculate the regional energy efficiency gap and investigate the impact of regional integration on the regional energy efficiency gap through the generalized moment method (GMM). The results show that 1) average city-level energy efficiency is 44.2%, ranging from 2.9% to 75.5%, indicating that China has a huge regional energy efficiency gap; 2) there is a U-shaped relationship between regional integration and the regional energy efficiency gap within city agglomeration. Improvement in regional integration can narrow the regional energy efficiency gap when the degree of regional integration is low, and expand the efficiency gap when regional integration level is high; 3) government intervention will smooth the impact of regional integration on the regional energy efficiency gap within city agglomeration. Practicable policies to mitigate the regional energy efficiency gap in China are suggested and applicable to other emerging economies, especially for those with a huge imbalance in regional energy efficiency.