With the rapid increasing of wind power generation in the power system, the coordinated dispatch of active and reactive power for each wind turbine (WT) in the wind farm (WF) becomes the critical issue for the safe and stable of power grid. Considering the time-varying characteristic of the WF, this can be regarded as a decision-making problem under uncertainty. To this end, this study formulates the active and reactive power dispatch problem of WF as a Markov decision process (MDP) allowing for the system uncertainty, e. g. wind speed, reactive power demand and wake effect. Then, an agent is trained via deep reinforcement learning algorithm (DRL) to solve the MDP to obtain the optimal dispatch policy with the minimizing levelized production cost (LPC) target. Finally, the proposed method is tested on an 80 MW WF and some benchmark methods are utilized to act as comparison examples. Simulation results show that, compared with other methods, the proposed dispatch strategy can provide more appropriate active and reactive reference for each wind turbine to extend lifetime of WF, resulting in less LPC.