In order to mitigate energy crisis and to meet carbon-emission reduction targets, the use of electrical energy produced by solar photovoltaic (PV) is inevitable. To meet the global increasing energy demand, PV power capacity will be expanded ranging from large-scale (from ten to several hundred MWs) PV farms at high and medium voltage level to kilowatt residential PV systems at low voltage level. It is expected that the PV penetration will increase in power systems with the retirement of traditional carbon-emission emitting power plants. Solar energy is diurnal in nature and in practice, it is highly uncertain due to various perturbation effects. With the recent technological advancements and rapid cost reductions in electrical energy storage (EES), EES could be deployed to enhance the system's performance and stability. This paper presents a comprehensive review on the emerging high penetration of PV with an overview of EES for PV systems. The crucial element, the building block of solar panel and the solar cell are reviewed. The emerging cell technologies are presented. A study of solar power forecasting techniques, an important tool for the successful operation and planning of PV and EES is included. A selection of EES is presented and studied for PV system purposes. Future research and areas for improvements in recent works and related areas are identified.