Abstract Since high temperature superconducting magnetic energy storage system (HT SMES) has attracted significant attention for their fast response in milliseconds, high efficiency (cyclic efficiency over 95%) and unlimited times of charging and discharging cycles, it can be used for system stabilizing – damping out low frequency power oscillations. A voltage source converter (VSC) based HTS SMES has been optimal designed for achieving a high efficiency and has been constructed by China Electric Power Research Institute (CEPRI). This SMES can store the maximum energy, while for the first time used two states of art high temperature superconductors, YBCO and BSCCO tapes. It has been tested in a 110 kV transmission power system by a dynamic power fluctuation compensation experiment using three different controlling strategies in CEPRI. The experimental output powers with these three strategies are compared and the results show that the SMES can trace the power variation and provide the required power to restrain the power fluctuation in milliseconds successfully. Finally, the application planning of SMES with the equivalent capacity in a practical renewable power system at Zhangbei wind power test base is evaluated by a case study based on the PSCAD/EMTDC simulation. An optimal switch time of the SMES in wind power system is presented using the real transmission parameters of Zhangbei power grid. This study can provide a reference for the demonstration of large-scale SMES systems in renewable power system.