Protection from soil erosion and efficient use of water are vital to sustainable dryland potato production in semi-arid regions. Introducing legumes into semi-arid agricultural systems as intercrops improves soil quality due to biological nitrogen fixation and reduced wind erosion as a result of better soil cover, but the consequences of introducing legumes for the water use efficiency of the crop system are less predictable. Here we carried out field experiments from 2014 to 2017 in Inner Mongolia, China. We compared a rotational intercropping system of potato (Solanum tuberosum L.) and hairy vetch (Vicia villosa) with monocultures of potato and hairy vetch and quantified crop yield, water use efficiency and land productivity. While the relative density (the ratio of plant density in intercrop and the density in sole stand) of both crops in the intercropping was 0.5, the average relative yield of the potato over four years was 0.43, but that of the vetch was 0.87, indicating dominance of the vetch in the intercropping system. Land and water equivalent ratios, defined as the area of land or amount of water that would be needed in single cropping to achieve the same yield as in intercropping, averaged to 1.30 and 1.29 over the years, respectively, indicating high relative land and water productivity of potato/hairy vetch intercropping compared to monocultures. Vetch was a stronger competitor for water than potato with a partial water equivalent ratio of 0.83. We conclude that the potato/vetch intercropping system improves land productivity and system level water use efficiency under the rain-fed semi-arid conditions of the study site. These results are useful to optimize cropping systems for regional sustainability with consideration of both arable crop production (potato) and provision of fodder for animal husbandry (vetch).