Deteriorating air quality is one of the most important environmental factors posing significant health risks to urban dwellers. Therefore, an exploration of the factors influencing air pollution and the formulation of targeted policies to address this issue are critically needed. Although many studies have used semi-parametric geographically weighted regression and geographically weighted regression to study the spatial heterogeneity characteristics of influencing factors of PM2.5 concentration change, due to the fixed bandwidth of these methods and other reasons, those studies still lack the ability to describe and explain cross-scale dynamics. The multi-scale geographically weighted regression (MGWR) method allows different variables to have different bandwidths, which can produce more realistic and useful spatial process models. By applying the MGWR method, this study investigated the spatial heterogeneity and spatial scales of impact of factors influencing PM2.5 concentrations in major Chinese cities during the period 2005–2015. This study showed the following: (1) Factors influencing changes in PM2.5 concentrations, such as technology, foreign investment levels, wind speed, precipitation, and Normalized Difference Vegetation Index (NDVI), evidenced significant spatial heterogeneity. Of these factors, precipitation, NDVI, and wind speed had small-scale regional effects, whose bandwidth ratios are all less than 20%, while foreign investment levels and technologies had medium-scale regional effects, whose bandwidth levels are 23% and 32%, respectively. Population, urbanization rates, and industrial structure demonstrated weak spatial heterogeneity, and the scale of their influence was predominantly global. (2) Overall, the change of NDVI was the most influential factor, which can explain 15.3% of the PM2.5 concentration change. Therefore, an enhanced protection of urban surface vegetation would be of universal significance. In some typical areas, dominant factors influencing pollution were evidently heterogeneous. Change in wind speed is a major factor that can explain 51.6% of the change in PM2.5 concentration in cities in the Central Plains, and change in foreign investment levels is the dominant influencing factor in cities in the Yunnan-Guizhou Plateau and the Sichuan Basin, explaining 30.6% and 44.2% of the PM2.5 concentration change, respectively. In cities located within the lower reaches of the Yangtze River, NDVI is a key factor, reducing PM2.5 concentrations by 9.7%. Those results can facilitate the development of region-specific measures and tailored urban policies to reduce PM2.5 pollution levels in different regions such as Northeast China and the Sichuan Basin.