With China’s rapid economic growth, energy-related CO2 emissions have experienced a dramatic increase. Quantification of energy-related CO2 emissions that occur in China is of serious concern for the policy makers to make efficient environmental policies without damaging the economic growth. Examining 33 productive sectors in China, this paper combined the extended “Kaya identity” and “IPAT model” with the Log-Mean Divisia Index Method (LMDI) to analyze the contribution of various factors driving of energy-related CO2 emissions in China during 1995–2009. Empirical results show that the main obstacle that hinders China’s transition to a green energy economy is the economic structure characterized by high carbon emissions. In contrast, the increased proportion of renewable energy sources (RES) and the improvement of energy efficiency play a more important role in reducing carbon emissions. Moreover, the power sector has a pivotal position in CO2 emissions reduction, primarily because of the expansion of electricity consumption. These findings suggest that policies and measures should be considered for various industrial sectors to maximize the energy efficiency potential. In addition, optimizing the industrial structure is more urgent than adjusting the energy structure for China.