Abstract Background The African continent is known for high entrepreneurial activity, especially in the agricultural sector. Despite this, the continent's economic development is below expectations, due to numerous factors constraining the growth and sustainability of agricultural SMEs. These constraints have been exacerbated by the COVID-19 pandemic. The purpose of this study was to understand the pathways through which the pandemic affected agri-SMEs, with specific focus on assessing the differentiated effects arising from the size of the agri-SME and the gender of the owner-manager. Methods Data was collected from over 100 agri-SMEs, ranging in size from sole proprietorships with one employee to agri-SMEs employing up to 100 people, in six African countries. Mixed methods were used to analyse the data with changes in business operations arising from changing market access, regimented health and safety guidelines and constrained labour supply assessed using visualisations and descriptive statistics. Logistic regression modelling was employed to determine the set of variables contributing to agri-SME business downturn during the COVID-19 pandemic. Results All surveyed agri-SMEs were negatively affected by COVID-19-associated restrictions with the size of the firm and gender of the owner-managers resulting in differentiated impacts. The smallest agri-SMEs, mainly owner-managed by women, were more likely to experience disruptions in marketing their goods and maintaining their labour supply. Larger agri-SMEs made changes to their business operations to comply with government guidelines during the pandemic and made investments to manage their labour supply, thus sustaining their business operations. In addition, logistic regression modelling results show that financing prior to the pandemic, engaging in primary agricultural production, and being further from urban centres significantly influenced the likelihood of a firm incurring business losses. Conclusions These findings necessitate engendered multi-faceted agri-SME support packages that are tailored for smaller-sized agri-SMEs. Any such support package should include support for agri-SMEs to develop sustainable marketing strategies and help them secure flexible financing that considers payment deferrals and debt moratorium during bona fide market shocks such as the COVID-19 pandemic.