The liquified natural gas (LNG) is currently considered an attractive marine fuel in the short- to medium-terms that can lead to the reduction of the shipping industry carbon emissions. LNG fuelled ocean-going ships have been designed by employing either low-pressure or high-pressure fuel systems. This study aims at enhancing the safety of a high-pressure fuel gas supply system (FGSS) designed for ocean-going LNG fuelled vessels. A model-based safety analysis is performed by employing the MADe™ software. The functional model of the baseline design of the investigated system is developed and subsequently employed to carry out the Failure Modes, Effects and Criticality Analysis (FMECA), as well as the quantitative Fault Tree Analysis (FTA). FMECA provided the risk priority number (RPN) for the identified system failure scenarios, the analysis of which leads to the identification of the FGSS critical components. The FTA results, which include the probabilities of the selected top events and appropriate importance metrics, are used for the FMECA results verification. This study results demonstrate that the FGSS critical components include the process valves responsible for supplying either LNG or natural gas to the gas treatment system and consumers respectively, as well as the pressure feedback controllers. Recommendations for design alterations pertain to the addition of pressure sensors and redundancy of the identified critical system components. The derived results demonstrate that impact of these design alterations on the calculated safety metrics are quite considerable, thus enhancing the safety of the baseline design. This study contributes to the safety analysis of FGGS and supports decisions for the designer and operator of LNG fuelled ships.