An accelerated loss of life (LOL) of distribution transformers has been observed in recent years owing to the increasing penetration of electric vehicles (EVs). This paper proposes an evolutionary curriculum learning (ECL)-based multi-agent deep reinforcement learning (MADRL) approach for optimizing transformer LOL while considering various charging demands of different EV owners. Specifically, the problem of charging multiple EVs is cast as a Markov game. It is solved by the proposed MADRL algorithm by modeling each EV controller as an agent with a specific objective. During the centralized training stage, a novel centralized ECL mechanism is adopted to enhance the coordination of multiple EVs. It enables the proposed approach to address the management of large-scale EV charging. When the training procedure is completed, the proposed approach is deployed in a decentralized manner. Herein, all the agents make decisions based solely on local information. The decentralized manner of execution helps preserve the privacy of EV owners, reduce the related communication cost, and avoid single-point failure. Comparative tests utilizing real-world data demonstrate that the proposed approach can achieve coordinated charging of a large number of EVs while satisfying the various charging demands of different EV owners.