Grid-connected hybrid renewable power systems with energy storage can reduce the intermittency of renewable power supply. However, emerging energy storage technologies need improvement to compete with lithium-ion batteries and reduce the cost of energy. Identifying and optimizing the the most valuable improvement path of these technologies is challenging due to the non-linearity of the energy system model when considering parameters as independent variables. To overcome this, a novel investment-based optimization method is proposed. The method involves linear optimization of the hybrid renewable energy system and subsequent investment optimization, accounting for diminishing improvements per investment. Applied to thermal energy, pumped thermal energy, molten salt, and adiabatic compressed air energy storage technologies, the results show that enhancing discharge efficiency is most valuable for all technologies. Reducing discharge capacity costs and energy storage capacity cost can also become important. Charge capacity cost and charge efficiency are found to be of lesser significance. The study provides detailed improvement pathways for each technology under various operational conditions, assisting developers in resource allocation. Overall, the investment-based optimization method and findings contribute to enhancing the competitiveness of emerging energy storage technologies and reducing reliance on batteries in renewable energy systems.