Satellite fire products have the potential to construct inter‐annual time series of fire activity, but estimating area burned requires considering biases introduced by orbiting geometry, fire behavior, and the presence of clouds and smoke. Here we evaluated the performance of fire counts from the Advanced Thermal Scanning Radiometer (ATSR) for the boreal forest region using area burned information from other sources. We found ATSR detection rate varied between regions and different years, being higher during large fire years than during small fire years. The results show ATSR fire counts do not represent an unbiased sample of fire activity, and independent validation may be required prior to using this data set in studies of global emissions from biomass burning.