Abstract Livestock grazing at low stocking rates is widely recommended to maintain grassland biodiversity. However, empirical evidence of grazing‐intensity effects on plant diversity is contradictory. Explicitly considering the small‐scale heterogeneity of short, frequently grazed and tall, rarely grazed patches typical of low‐input grazing systems may be crucial to the understanding of paddock‐scale grazing effects. We studied three patch types (short, intermediate, tall) within an unfertilised long‐term cattle grazing experiment in Lower Saxony, Germany, comparing three paddock‐scale grazing intensities. We analysed soil nutrient concentrations and recorded vegetation composition at a total of 135 plots. We determined species richness, Simpson diversity, Simpson evenness and beta diversity of individual plots (plot scale) and patch types within paddocks (patch scale). To quantify paddock‐scale diversity, we resampled plot‐scale species composition across a gradient of relative proportions of short and tall patches within a paddock. Patch type, not paddock‐scale grazing intensity, was the main driver of plant diversity at both plot and patch scale. Short patches were more diverse than tall patches, but the effect was not strongly mediated by the lower soil nutrient concentrations in the short patches. By contrast, both patch type and grazing intensity affected vegetation composition at plot and patch scale. Beta‐diversity within and between patch types was independent of grazing intensity; consequently, paddock‐scale diversity was determined by the relative proportion of short versus tall patches. Higher alpha diversity of short patches compared to tall patches was more important than beta diversity between the two patch types in shaping paddock‐scale diversity. Consequently, with increasing short‐patch proportion, paddock‐scale diversity increased. Synthesis and applications. Our study identifies the grazing‐induced patch structure as the most important driver of plant diversity across different grazing intensities in low‐input, that is, unfertilised and continuously stocked, pastures. To optimise grazing management for biodiversity, understanding plant‐diversity responses to grazing at the patch scale is indispensable. Our results suggest that, in unfertilised, continuously stocked European pastures, trade‐offs between biodiversity and agronomic production may be small, as short patches, whose proportion increases with stocking rate, also had the highest plant diversity.