The increasing number of decentralized renewable energy sources together with the grow in overall electricity consumption introduce many new challenges related to dimensioning of grid assets and supply-demand balancing. Approximately 40% of the total energy consumption is used to cover the needs of commercial and office buildings. To improve the design of the energy infrastructure and the efficient deployment of resources, new paradigms have to be thought up. Such new paradigms need automated methods to dynamically predict the energy consumption in buildings. At the same time these methods should be easily expandable to higher levels of aggregation such as neighbourhoods and the power distribution grid. Predicting energy consumption for a building is complex due to many influencing factors, such as weather conditions, performance and settings of heating and cooling systems, and the number of people present. In this paper, we investigate a newly developed stochastic model for time series prediction of energy consumption, namely the Conditional Restricted Boltzmann Machine (CRBM), and evaluate its performance in the context of building automation systems. The assessment is made on a real dataset consisting of 7 weeks of hourly resolution electricity consumption collected from a Dutch office building. The results showed that for the energy prediction problem solved here, CRBMs outperform Artificial Neural Networks (ANNs), and Hidden Markov Models (HMMs).