With many favorable advantages including fast response ability in particular, utility-level energy storage systems (ESS) are being integrated into energy and reserve markets to help mitigate uncertain renewable resources and fluctuant demands. This paper discusses a stochastic unit commitment (UC) model to explore capabilities of ESSs in providing valuable grid services by simultaneously joining energy and reserve markets. The proposed reserve model of ESSs presents the following features: (i) two constraints are proposed to formulate ESS’s reserve provision ability in each hour via six operation modes, namely increasing/reducing the level of charge, switching to discharge, increasing/reducing the level of discharge, switching to charge; (ii) as an energy-limited asset, constraints on ESS’s reserve deliverability across multiple hours is introduced to hold enough state of charge (SOC) headroom and floor room, guaranteeing that reserves cleared in individual hours are continuously deliverable over multiple successive hours without violating operation limits. Moreover, a scenario-based UC model, which addresses uncertainties of renewables and demands, is adopted to evaluate reserves provided by ESSs and generators. To effectively solve the stochastic UC problem, the progressive hedging algorithm with heuristic approaches is discussed. A 6-bus system and a modified IEEE 118-bus system are used to illustrate effectiveness of the proposed approaches.