Dry-type air-core reactor used as AC filter reactors is one of the main noise sources in HVDC converter stations. The research on the vibration characteristics of the dry-type air-core reactor is of great significance for the design of noise reduction measures. Based on the assumption of linear vibration system, IEC 60076-6:2007 suggests to use sum frequency, difference frequency and double frequency of the loading current frequency as the decomposition of the total excitation, to measure the acoustic sound level of dry-type air-core reactor. To verify the assumption, this paper aims at the nonlinearity of the dry-type air-core reactor vibration system. The radical vibration of the tested reactor was firstly measured by a laser Doppler vibrometer. The measurement results show that: under single-frequency current, except for the double frequency, the vibration spectrum also contains quadruple frequency and the sixth harmonic frequency components; while under multiple frequency current, other frequency components also exist besides the sum, difference and double components of the current frequency. The ultra harmonic phenomenon is a typical feature of nonlinear systems. Then the vibration equation was also theoretically established. Considering the coupling of magnetic field and vibration, the perturbation method is used to solve the vibration equation. The theoretical analysis results are consistent with the experiments. It is concluded that the nonlinearity of the vibration system of dry-type air-core reactors is not negligible, and one cause of the nonlinearity is the coupling of the magnetic field and the motion of the reactor.