A square cylinder with a V-shaped groove on the windward side in the piezoelectric cantilever flow-induced vibration energy harvester (FIVEH) is presented to improve the output power of the energy harvester and reduce the critical velocity of the system, aiming at the self-powered supply of low energy consumption devices in the natural environment with low wind speed. Seven groups of galloping piezoelectric energy harvesters (GPEHs) were designed and tested in a wind tunnel by gradually changing the angle of two symmetrical sharp angles of the V-groove. The GPEH with a sharp angle of 45° was selected as the optimal energy harvester. Its output power was 61% more than the GPEH without the V-shaped groove. The more accurate mathematical model was made by using the sparse identification method to calculate the empirical parameters of fluid based on the experimental data and the theoretical model. The critical velocity of the galloping system was calculated by analyzing the local Hopf bifurcation of the model. The minimum critical velocity was 2.53 m/s smaller than the maximum critical velocity at 4.69 m/s. These results make the GPEH with a V-shaped groove (GPEH-V) more suitable to harvest wind energy efficiently in a low wind speed environment.