Feature selection helps improve the accuracy and computational time of solar forecasting. However, FS is often passed by or conducted with methods that do not suit the solar forecasting issue, such as filter or linear methods. In this study, we propose a wrapper method termed Sequential Forward Selection (SFS), with a Kernel Conditional Density Estimator (KCDE) named SFS-KCDE, as FS to forecast day-ahead regional PV power production in French Guiana. This method was compared to three other FS methods used in earlier studies: the Pearson correlation method, the RReliefF (RRF) method, and SFS using a linear regression. It has been shown that SFS-KCDE outperforms other FS methods, particularly for overcast sky conditions. Moreover, Wrapper methods show better forecasting performance than filter methods and should be used.