This study investigates the flutter response of a rectangular cylinder model with an aspect ratio of 5 at the Reynolds number Re = 100 via direct numerical simulation. The effects of two key parameters, i.e., the moment of inertia and reduced flow velocity, on the aerodynamic performance and dynamic responses of the cylinder in the state of torsional flutter are investigated. To reveal the flutter mechanism, the high-order dynamic mode decomposition (HODMD) analysis is conducted to decompose the flow field. The results show that both an increase in the moment of inertia and a higher reduced flow velocity lead to a larger torsional amplitude and a corresponding decrease in torque. At the same time, the primary frequency decreases and the size of the shedding vortex gradually enlarges. The vortices shed from the leading edge and the trailing edge of the model form a 2P wake pattern. The leading-edge vortex is significantly larger than the trailing-edge vortex in terms of strength and size. The leading edge plays a dominant role and only contributes to the odd-order HODMD modes while the even-order modes are deemed inconsequential. As the moment of inertia increases, the total energy of the higher-order modes increases, which has the same results as the power spectral density of torque, reflecting increased nonlinearity and complexity of the system. Similarly, increasing the reduced flow velocity at the same moment of inertia has similar excitation effects.