The uncertain nature of renewable energy resources (RERs) and fast demand response lead to recurring voltage violations in the power systems, which causes frequent transformer tap shifting and capacitor switching. Therefore, this paper resorts to the static var compensators (SVCs) to manage the bus voltages based on the multi-agent deep reinforcement learning (MA-DRL) algorithm. The proposed scheme includes several system agents and SVC agents to collaboratively adjust the injected reactive power to restrict the bus voltages within the normal range. All the agents are trained centrally and executed separately, which requires minimum communication cost. The IEEE 14-bus system, IEEE 300-bus system, and China 157-node urban power grid are used to verify the effectiveness of the proposed method.