Abstract With the increasing momentum towards flexible power systems based on renewables, the role of hydropower has great importance, especially for providing balancing power. In this paper, a fundamental study on the operating stability of hydropower generating systems is conducted to reveal the practical characteristics for the newly emergent issue of ultra-low frequency oscillations. A unique study methodology is adapted by combing the theoretical analysis and the physical model experiment. In this paper, first, the set-up of the integral experiment platform for the transient processes of the pumped storage plants is presented. Second, a mathematical model of hydropower generating systems is built, and the theoretical stability analysis is conducted based on the Routh-Hurwitz criterion and the stability margin region. The model experiments related to the frequency stability of hydropower generating systems were conducted with reference to the stability region from theoretical analysis. The results demonstrate the sustained ultra-low frequency oscillations and frequency instability of hydropower units in experiments for the first time. Attenuation characteristics of the oscillations are theoretically derived based on the stability margin region, and then quantitatively identified by experiments. The experiment accorded with theoretical stability region within a reasonable tolerance that corresponded to the ±0.1 stability margin.