Supplementing the diet of ruminants with agro-industrial by-products is a common practice. In this study, we applied an untargeted lipidomics approach to study the changes in the milk lipid metabolite profiles linked to the addition of different doses of spent coffee grounds (SCG) to the diet of lactating goats. The carryover of caffeine from feed to milk was also studied. Compared to controls, the milk of goats on the SCG diet showed higher levels of cholesteryl esters, sphingomyelins, and phospholipids, while nonesterified fatty acids were downregulated. After 12 h from the last SCG dose, the carry-over of caffeine was, on average, 3%. Collectively, our results establish that SCG supplementation induces changes in the milk levels of complex lipid molecules and causes the transfer of caffeine and caffeine metabolites from feed to milk.