Knowing the extent and environmental drivers of forests is key to successfully restore degraded ecosystems, and to mitigate climate change and desertification impacts using tree planting. Water availability is the main limiting factor for the development of forests in drylands, yet the importance of groundwater resources and palaeoclimate as drivers of their current distribution has been neglected. Here we report that mid-Holocene climates and aquifer trends are key predictors of the distribution of dryland forests worldwide. We also updated the global extent of dryland forests to 1,283 million hectares and showed that failing to consider past climates and aquifers has resulted in ignoring or misplacing up to 130 million hectares of forests in drylands. Our findings highlight the importance of a wetter past and well-preserved aquifers to explain the current distribution of dryland forests, and can guide restoration actions by avoiding unsuitable areas for tree establishment in a drier world.