Farmers worldwide have to deal with increasing climate variability and weather extremes. Most of the previous research has focused on impacts on agricultural production, but little is known about the related social and economic impacts on farmers. In this study, we investigated the social and economic impact of extreme weather events (EWE) on farmers in Nepal, and explored how they coped with and adapted to heat waves and cold spells between 2012 and 2017. To address these aims, we conducted a survey of 350 farms randomly selected from the Bardiya and Banke districts of the Terai lowlands of Nepal. They were specifically asked to rate the impacts of extreme temperatures, as well as their effect on labour productivity and collective farmer health, and the detailed preventative measures they had implemented. About 84% of the farmers self-reported moderate or severe heat stress during the last five years, and about 85%, moderate or severe cold stress. Likewise, the majority of respondents reported that both farmer health and labour productivity had been compromised by EWEs. Productivity loss had a strong association with the perceived levels of heat and cold stress, which, in turn, were more likely to be reported by farmers with previous EWE experience. Potentially due to the increased care required during EWEs, those farmers with livestock reported increased heat and cold stress, as, surprisingly, did those who had implemented adaptation measures. Farmers seemed to be less prepared for potential threats of cold spells than heat waves, and therefore less likely to adopt coping strategies, since these are a recent phenomenon. This study identified some limitations. The cross sectional and self-reported data, as a common source of information to estimate health impact, level of heat/cold stress and labour productivity loss. Community-based education/community engagement programs could be developed to facilitate proactive adaptation.