Nitrogen fertilization, irrigation and concentrate feeding are important factors in rotational pasture management for dairy farms in South Africa. The extent to which these factors affect environmental efficiency is subject to current and intense debate among scientists. A three-year field study was conducted to investigate the yield response of different N-fertilizer treatments (0 (N0), 220 (N20), 440 (N40), 660 (N60) and 880 (N80) kg N ha−1 year−1) on grazed pastures and to calculate the carbon footprint (CF) of milk produced. Excessive N-fertilization (N60 and N80) did not increase herbage dry matter and energy yields from pastures. However, N80 indicated the highest N-yield but at the same time also the highest N surpluses at field level. A maximum fertilizer rate of 220 kg ha−1 year−1 (in addition to excreted N from grazing animals) appears sufficient to ensure adequate herbage yields (~20 t DM ha−1 year−1) with a slightly positive field-N-balance. This amount will prevent the depletion of soil C and N, with low N losses to the environment, where adequate milk yields of ~17 t ECM ha−1 with a low CF (~1.3 kg CO2 kg ECM−1) are reached. Methane from enteric fermentation (~49% ± 3.3) and N2O (~16% ± 3.2) emissions from irrigated pastures were the main contributors to the CF. A further CF reduction can be achieved by improved N-fertilization planning, low emission irrigation techniques and strategies to limit N2O emissions from pasture soils in South Africa.