Governments contribute to the transition toward a circular economy (CE) by using criteria in their procurement processes that trigger the supply of circular products and services, namely circular public procurement (CPP). CPP practices are emerging in Europe. However, the effect of CPP is not yet monitored and hence remains unclear. What is the efficacy of CPP in reducing the impacts of goods and services? Analyzing CPP efficacy is an important next step in exploring how to improve its application. This paper presents the results of an effect evaluation of CPP in the Netherlands, using a sample-based mixed-method approach in combination with life cycle assessment for analyzing CPP-induced reduced impacts on global warming and material use. Two thirds of the procurement tenders in which circular procurement criteria were applied in 2017 and 2018 did not result in reduced environmental impacts or reduced material use. One third, however, showed that, as well as how CPP can contribute to the transition toward a CE. The identified remaining challenges are (1) to apply criteria that are ambitious enough to challenge the market and (2) to keep attention on the circularity ambitions up during the implementation phase of the procurement process. Effect indicators are proposed to complement the current monitoring practices of CPP implementation.