Participation of wind energy in the generation portfolio of power systems is increasing, making it more challenging for system operators to adequately maintain system security. It therefore becomes increasingly crucial to accurately predict the wind generation. This work investigates how different parameters influence the performance of forecasting algorithms. Firstly, this work analyzes the combined influence of the input data, batch size, number of neurons and hidden layers, and the training data on the forecast accuracy across forecast horizons of 5, 15, 30 and 60 min. It was found that increasing look ahead times require among others more hidden layers and lower batch sizes. Next, the optimizer and loss function leading to the most accurate forecasts were identified. It was concluded that the Adadelta optimizer and Mean Absolute Error loss function consistently result in the best performing forecasting algorithm. Finally, it was investigated if the most accurate optimizer-loss function combination is influenced by the choice of the performance metric. Whereas the Adadelta-Mean Absolute Error pair remains the most accurate combination irrespective of the evaluation metric, a strong relation was observed between the Root Mean Square Error performance metric and Mean Square Error loss function. Analyses were performed on 12 wind farms.