The perception of apparent sizes of buildings in a rural environment depends on the height ratio between the building and its surrounding vegetation, and it is this parameter which is currently used to assess the built landscapes. The impact of a contrasting height is less strong if the building does not exceed the horizon line. For buildings overshooting the skyline, the building’s level of sharpness and number of lines in contrast to the sky determines the impact of the scales, and vegetation in the background helps to reduce impact. The specific objectives of the present study were: (1) finding height–ratio thresholds between building and background vegetation, which may improve the integration of rural buildings in sky-sensitive locations, and; (2) comparing the results in two rural contexts with very different climatic conditions: Spain and Sweden. A survey of eighteen scenarios (nine Spanish and nine Swedish), all digitally modified with different relative height ratios between vegetation and buildings, was performed. The survey was evaluated by the public from both countries. Regardless of the country of origin, integration of the building was good or very good when the vegetation in background did not exceed one half of the height of the construction. These results may be translated to technical criteria for planning assessment.