Flowable fills are a type of fill material with many construction applications, including transportation engineering, building engineering, water conservancy constructions, etc. Flowable fills usually consist of cementing agents, water, and aggregates such as soils or other waste or cheap materials. Flowable fills have the characteristics of high flowability, self-leveling, self-compacting, high and adjustable strength, and the ability to adopt waste and cheap materials. In this study, a waste soil-based flowable fill was investigated under drying–wetting and freeze–thaw actions. Under six drying–wetting cycles, flowable fill specimens underwent a continuous reduction in strength, accompanying the mass losses and the changes in micro-structures. The level of strength reduction increased with decreased addition of Portland cement and increased addition of water. After six drying–wetting cycles, the specimens showed a 27–51% strength reduction as compared to their counterparts with no drying–wetting actions. Under freeze–thaw cycles, the specimens also showed noticeable but insignificant degradation. After six freeze–thaw cycles, the level of strength reduction ranged from 9–20%. Most of the strength reduction occurred during the first three cycles. Based on the test data, an empirical model was proposed to predict the strength reduction under drying–wetting cycles. The results proved that flowable fills may undergo a relatively large reduction in their engineering performance under adverse environments, especially drying–wetting actions. The implications of the results for construction are also discussed in the paper.