Traditional proportional integral (PI) control has been extensively used for power control of voltage source converters in microgrid systems. Previous studies show that fixed-gain PI controllers cannot easily adapt to power changes, disturbances, and parameters variation, especially in large microgrids; hence the need for continuous algorithms to adjust the controller gains over the transients cannot be neglected. In this paper, a novel online tuning algorithm for PI controllers is proposed and implemented in a microgrid system. In this algorithm, which is based on the neuro-dynamic programming concept, a fuzzy critic is employed to evaluate the credibility of the control system performance and provide an evaluation signal, which is then used in the gain-tuning process. The PI controller gains are updated in an optimization process based on steepest decent rule so that the evaluation signal produced by the critic is minimized. The developed control structure, which is named critic-based self-tuning PI controller, is tested in a microgrid system with different penetrations of distributed generators and operational scenarios. The simulation results verify that implementation of a heuristic gain-tuning algorithm results in a model-independent controller with increased adaptivity compared with conventional PI control. Furthermore, due to simple learning rules, the convergence time is significantly reduced and the transient response is improved. The proposed gain-tuning algorithm can also be applied to PI controllers in other applications of controllable systems.