For a sustainable building industry, reusable construction with a low demand for primary resources is needed. Moreover, if we want to reduce the amount of construction and demolition waste, construction with recycled aggregate should be considered. To investigate the environmental impacts of such concrete construction, life cycle assessment (LCA) was used to compare the following types of concrete construction: Reusable blocks with recycled brick aggregate, reusable blocks with recycled concrete, reusable blocks with natural aggregate, and regular concrete wall. Firstly, the properties of new concrete with recycled aggregate were measured, such as physical, mechanical, and thermal properties. Then, different constructions were designed and assessed using the method of Institute of Environmental Sciences (CML2001) and the method of National Institute for Public Health and the Environment (ReCiPe 2016) as characterization methods. Unsurprisingly, the regular concrete wall had a higher impact on most of the impact categories, e.g., 113 kg CO2 eq. (in the first scenario, using CML2001). In accordance with the circular principles, the reusability of blocks and recycling of aggregate are the main factors that affect the environmental impact of the constructions. Thus, the global warming potential (GWP) of construction with reusable recycled concrete blocks was only 53 kg CO2 eq. (in the second scenario). Moreover, we show differences in the results of CML2001 and ReCiPe 2016, e.g., in the Photochemical Oxidant Creation category.