The accumulation of soiling on photovoltaic modules and on the mirrors of concentrating solar power systems causes non-negligible energy losses with economic consequences. These challenges can be mitigated, or even prevented, through appropriate actions if the magnitude of soiling is known. Particle counting analysis is a common procedure to characterize soiling, as it can be easily performed on micrographs of glass coupons or solar devices that have been exposed to the environment. Particle counting does not, however, yield invariant results across institutions. The particle size distribution analysis is affected by the operator of the image analysis software and the methodology utilized. The results of a round-robin study are presented in this work to explore and elucidate the uncertainty related to particle counting and its effect on the characterization of the soiling of glass surfaces used in solar energy conversion systems. An international group of soiling experts analysed the same 8 micrographs using the same open-source ImageJ software package. The variation in the particle analyses results were investigated to identify specimen characteristics with the lowest coefficient of variation (CV) and the least uncertainty among the various operators. The mean particle diameter showed the lowest CV among the investigated characteristics, whereas the number of particles exhibited the largest CV. Additional parameters, such as the fractional area coverage by particles and parameters related to the distribution's shape yielded intermediate CV values. These results can provide insights on the magnitude inter-lab variability and uncertainty for optical and microscope-based soiling monitoring and characterization.