At the dawn of Industry 4.0, it has become apparent that assessment of engineered systems should be informed from the state of the system “as-is”. To this end, data needs to be fused with adequate and efficient system models. Such system models should account for the underlying physics and the possibly nonlinear dynamic processes involved. This paper introduces a physics-based parametric formulation for nonlinear structural systems. A Reduced Order Model (ROM) of the high fidelity system is developed, retaining the dependencies on system properties and on temporal and spectral characteristics of the excitation. The ROM formulation relies on i) Proper Orthogonal Decomposition applied to snapshots of the nonlinear response, and ii) manifold interpolation of the resulting projection bases. Its performance is evaluated on a 3D earthquake-excited shear frame with nonlinear couplings. The developed ROM can be exploited for a number of tasks including monitoring, diagnostics and residual life estimation of critical components.