Abstract Improving energy efficiency sustainability is a target of the Chinese government. However, the effectiveness of energy conservation policy is affected by the energy rebound effect under which energy efficiency improvement reduces the effective price of energy services, thereby completely or partially offsetting the energy saved by efficiency improvement. Based on the output distance function, this paper develops an improved estimation model of the energy rebound effect, which is logically consistent with the quantities of energy savings and energy rebounds induced by technological progress. Results show that the aggregate energy rebound effect of 36 industrial sectors in China over 1998–2011 is 88.42%, which implies that most of the expected energy savings are mitigated. Investment-driven economic growth is not conducive to energy-saving and results in a strong energy rebound effect in the following year. The equipment and high-end manufacturing sectors have low levels of rebound effect, indicating that increasing the proportion of such firms in the total manufacturing sector can improve the performance of energy conservation. The high level and heterogeneity in rebound effects strongly suggest that varies strategies are necessary for energy conservation among China’s industrial sectors.