Nature gas hydrates (NGHs) are considered as an ideal substitute for traditional fossil fuel. The evaluation of the physical properties of hydrate-bearing sediment (HBS) is important to develop reasonable exploitation strategies and clarify the corresponding main controlling factors. In this study, to determine the influence of grain size distribution on the physical properties of HBSs, 24 hydrate-bearing models were reconstructed based on a novel pore-scale 3D morphological modeling algorithm. A series of pore system evaluations and permeability simulations were carried out by pore network modeling (PNM). Moreover, thermal and electrical simulations were carried out using the finite volume method (FVM). The results are as follows: (1) The smaller the grain size is, the smaller the radius of the pores and throats of HBSs will be, which makes it easier for the connectivity between pores and throats to deteriorate as the hydrate saturation increases. (2) With increasing hydrate saturation, the permeability of HBSs gradually decreases. According to the decline rate, it can be divided into two stages: in the first stage, it will decrease sharply, while in the second stage, it will decrease slowly. In general, the smaller the grain size is, the lower the permeability of HBSs will be. (3) The apparent thermal conductivity of HBSs will increase as the hydrate saturation increases. The smaller the grain size is, the higher the apparent thermal conductivity will be. (4) The smaller the grain size is, the lower the apparent electrical conductivity and the higher the resistivity index under the same hydrate saturation will be.