The large-scale grid connection of new energy wind power generation has caused serious challenges to the power quality of the power system. The hybrid energy storage system (HESS) is an effective means to smooth the fluctuation of wind power and improve the economy of the system. In order to determine the optimal capacity configuration of the hybrid energy storage system, first, a decomposition method which combines ensemble empirical mode decomposition (EEMD) and empirical mode decomposition (EMD) is proposed, and a series of intrinsic mode functions are obtained, the grey correlation analysis method is used to analyze the similarity, and the components with similar correlation values are reconstructed to obtain high-frequency and low-frequency components; second, considering the battery life loss of the hybrid energy storage system, with the goal of minimizing the entire life cycle cost, the optimal configuration model of hybrid energy storage capacity is established, and different energy storage schemes are analyzed to obtain the energy storage configuration scheme with the best economy; finally, based on the typical daily historical data of a wind farm, the effectiveness and economy of the proposed method are verified.