Energy management is an enabling technique to guarantee the reliability and economy of hybrid electric systems. This paper proposes a novel machine learning-based energy management strategy for a hybrid electric bus (HEB), with an emphasized consciousness of both thermal safety and degradation of the onboard lithium-ion battery (LIB) system. Firstly, the deep deterministic policy gradient (DDPG) algorithm is combined with an expert-assistance system, for the first time, to enhance the “cold start” performance and optimize the power allocation of HEB. Secondly, in the framework of the proposed algorithm, the penalties to over-temperature and LIB degradation are embedded to improve the management quality in terms of the thermal safety enforcement and overall driving cost reduction. The proposed strategy is tested under different road missions to validate its superiority over state-of-the-art techniques in terms of training efficiency and optimization performance.