Climate change is estimated to exacerbate existing challenges faced by smallholder farmers in Sub-Sahara Africa. However, limited studies quantify the extent of variation in climate change impact under these systems at the local scale. The Decision Support System for Agro-technological Transfer (DSSAT) was used to quantify variation in climate change impacts on maize yield under current agricultural practices in semi-arid regions of Senegal (Nioro du Rip) and Ghana (Navrongo and Tamale). Multi-benchmark climate models (Mid-Century, 2040–2069 for two Representative Concentration Pathways, RCP4.5 and RCP8.5), and multiple soil and management information from agronomic surveys were used as input for DSSAT. The average impact of climate scenarios on grain yield among farms ranged between −9% and −39% across sites. Substantial variation in climate response exists across farms in the same farming zone with relative standard deviations from 8% to 117% at Nioro du Rip, 13% to 64% in Navrongo and 9% to 37% in Tamale across climate models. Variations in fertilizer application, planting dates and soil types explained the variation in the impact among farms. This study provides insight into the complexities of the impact of climate scenarios on maize yield and the need for better representation of heterogeneous farming systems for optimized outcomes in adaptation and resilience planning in smallholder systems.