Olea europaea L. is a crop typical of the Mediterranean area that has an important role in economy, society, and culture of this region. Climate change is expected to have significant impact on this crop, which is typically adapted to certain pedo-climatic characteristics of restricted geographic areas. In this scenario, the aim of this study was to evaluate the time-course response of pollen viability to different combinations of temperature and humidity. The study was performed comparing flowering time and pollen functionality of O. europaea from twelve cultivars growing at the same site belonging to the Campania olive collection in Italy. Pollen was incubated at 12 °C, 22 °C, and 36 °C in combination with 50% RH or 100% RH treatments for 5 days. The results highlighted that a drastic loss of pollen viability occurs when pollen is subjected to a combination of high humidity and high temperature, whereas 50% RH had less impact on pollen thermotolerance, because most cultivars preserved a high pollen viability over time. In the ongoing climate change scenario, it is critical to assess the effect of increasing temperatures on sensitive reproductive traits such as pollen viability to predict possible reduction in crop yield. Moreover, the results highlighted that the effect of temperature increase on pollen thermotolerance should be evaluated in combination with other environmental factors such as humidity conditions. The screening of olive cultivars based on pollen thermotolerance is critical in the ongoing climate change scenario, especially considering that the economic value of this species relies on successful fertilization and embryo development, and also that production cycle of Olea europaea can be longer than a hundred years.