This study examined the influence of heat recovery ventilators (HRVs) on energy savings and indoor air quality (IAQ) in high-rise residential buildings. Field measurements were performed in four residential units, which were validated by computer simulations and estimated the total annual energy consumption. The operation schedules for HRVs were determined by a survey of residents. Field measurement results indicate that HRVs could effectively improve IAQ and afford effective energy savings. The indoor concentrations of formaldehyde were reduced by 54.6% after HRVs were operated for 24 h. The initial concentration was reduced by 82% after 168 h. Toluene was the dominant volatile organic compounds (VOCs) in the indoor air. Its initial concentration was reduced by 50% and other VOCs were also reduced by 40.1% to 53.1% after HRVs were operated. Annual energy savings of up to 20.26% were predicted when HRVs were operated for 24 h continuously, exchanging sensible and latent heat. HRVs could save energy more effectively in winter than in summer due to the greater temperature difference between outdoor and indoor air. Based on the preferred operation schedules of homes surveyed, an annual energy savings could be as high as 8.52%.