As a consequence of a fast-paced technological evolution along with the acknowledgment of utilizing clean and renewable energy resources over fossil fuels, the importance of energy storage devices is widely recognized. The electrochemical capacitor (EC), commonly known as a supercapacitor or ultracapacitor, is an energy storage device that is already being used in portable consumer electronics, electrification of transportation, and grid-level applications. High power density and long cycle life are the two most prominent properties of ECs, thanks to the electrostatic nature of their charge storage mechanism. These properties are well utilized in a system where ECs are used as a backup power-boosting device to rechargeable batteries. By providing the peak power required, they eventually prolong the battery lifetime. However, the relatively low energy density of ECs compared to rechargeable batteries limits their application as a standalone device. In addition, low operating voltage, adverse self-discharge rate, severe leakage current, elevated temperature incompatibility are some of the crucial issues that are preventing the widespread application of ECs. Besides a general discussion about ECs, the main objective of this thesis is to identify and address the above-mentioned critical challenges, and to propose and demonstrate corresponding solutions. Firstly, it is revealed that utilizing a redox-active KBr electrolyte can enhance both operating voltage and capacitance, and hence increases energy density without sacrificing power density or cycle life. Secondly, an evaluation of elevated temperature influence on the capacitive performance of ECs containing ionic liquid (IL) electrolyte demonstrates a high working temperature beyond 120 °C. Thirdly, a systematic investigation of ECs containing IL at elevated temperatures shows a significant increase of the self-discharge rate with temperature and pinpoints the underlying mechanisms; at lower initial voltages the self-discharge rate is dominated by diffusion of electrolyte ions rather than charge redistribution. Fourthly, the addition of a small amount of liquid crystals (LC) in neutral electrolyte shows a reduction of self-discharge and leakage current due to slower diffusion of ions in the device, which is proposed to originate from the anisotropic properties of LC. Finally, by utilizing the thermocapacitive effect, a thermal charging of ECs containing IL is demonstrated, where a high voltage of more than 900 mV could be recovered when two devices in series are exposed to a 60 °C temperature environment.