Over 100 gigatons of terrestrial plant biomass are produced globally each year. Ninety percent of this biomass escapes herbivory and enters the dead organic matter pool, thus supporting complex detritus-based food webs that determine the critical balance between carbon mineralization and sequestration. How will changes in biodiversity affect this vital component of ecosystem functioning? Based on our analysis of concepts and experiments of leaf decomposition in forest floors and streams, we suggest that changes in species diversity within and across trophic levels can significantly alter decomposition. This happens through various mechanisms that are broadly similar in forest floors and streams. Differences in diversity effects between these systems relate to divergent habitat conditions and evolutionary trajectories of aquatic and terrestrial decomposers.