Overfishing and habitat degradation through climate change pose the greatest threats to sustainability of marine resources on coral reefs. We examined how changes in fishing pressure and benthic habitat composition influenced the size spectra of island‐scale reef fish communities in Lau, Fiji. Between 2000 and 2006 fishing pressure declined in the Lau Islands due to declining human populations and reduced demand for fresh fish. At the same time, coral cover declined and fine‐scale architectural complexity eroded due to coral bleaching and outbreaks of crown‐of‐thorns starfish, Acanthaster planci. We examined the size distribution of reef fish communities using size spectra analysis, the linearized relationship between abundance and body size class. Spatial variation in fishing pressure accounted for 31% of the variation in the slope of the size spectra in 2000, higher fishing pressure being associated with a steeper slope, which is indicative of fewer large‐bodied fish and/or more small‐bodied fish. Conversely, in 2006 spatial variation in habitat explained 53% of the variation in the size spectra slopes, and the relationship with fishing pressure was much weaker (∼12% of variation) than in 2000. Reduced cover of corals and lower structural complexity was associated with less steep size spectra slopes, primarily due to reduced abundance of fish <20 cm. Habitat degradation will compound effects of fishing on coral reefs as increased fishing reduces large‐bodied target species, while habitat loss results in fewer small‐bodied juveniles and prey that replenish stocks and provide dietary resources for predatory target species. Effective management of reef resources therefore depends on both reducing fishing pressure and maintaining processes that encourage rapid recovery of coral habitat.