This paper proposes a novel multi-agent deep reinforcement learning (MADRL) approach for the energy management of multiple microgrids considering the robust voltage control under the missing measurements. Missing measurement control poses challenges to the MADRL. To address the problem, we propose a trajectory history information-utilized opponent modeling-based distributed MADRL to avoid the collapse of control caused by the loss of current time measurement. Simulation results demonstrate that, whether the measurements are complete or not, the proposed approach achieves the ideal results.