Energy security is an important policy goal for most countries. Here, we show that cross-country differences in concern about energy security across Israel and 22 countries in Europe are explained by energy-specific and general national contextual indicators, over-and-above individual-level factors that reflect population demographics. Specifically, public concerns about import dependency and affordability reflect the specific energy context within countries, such as dependency on energy imports and electricity costs, while higher concerns about the affordability, vulnerability and reliability of energy are associated with higher fossil fuel consumption. More general national context beyond energy also appears to matter; energy security concerns are higher in countries that are doing less well in terms of economic and human well-being. These findings indicate that wider energy, social and economic context influence people's feelings of vulnerability and sense of security, which may inform the development of effective energy security strategies that assuage public concerns.