Abstract Accurate generation forecasting can effectively accelerate the use of renewable energy in hybrid energy systems, contributing significantly to the delivery of the net-zero emission target. Recently, neural-network-based quantile forecast models have shown superior performance on renewable energy generation forecasting, partially because they have subtly embedded quantile forecast evaluation metrics into their loss functions. However, the non-differentiability of involved metrics has rendered their metric-embedded loss functions not everywhere-derivable, resulting in inapplicability of gradient-based training approaches. Instead, they have resorted to heuristic searches for Neural Network (NN) training, bringing low training efficiency and a rigid restriction on the size of the resultant NN. In this paper, the Indicator Gradient Descent (IGD) is proposed to overcome the non-differentiability of involved metrics, and several metric-embedded loss functions are innovatively customized combining IGD, enabling NNs to be trained efficiently in a ‘gradient-descent-like’ manner. Moreover, the deep Bidirectional Long Short-Term Memory (BiLSTM) is adopted to capture the periodicity of renewable generation (diurnal and seasonal patterns), and the residual technique is used to improve the training efficiency of the deep BiLSTM. Finally, a Deep Quantile Forecast Network (DQFN) based on IGD and deep residual BiLSTM is developed for wind and solar power quantile forecasting. Practical experiments in four cases have verified the effectiveness and efficiency of DQFN and IGD, where DQFN has achieved the lowest average proportion deviations (all below 1.7%) and the highest skill scores.