We consider the random fluctuations of the free energy in the $p$-spin version of the Sherrington-Kirkpatrick model in the high temperature regime. Using the martingale approach of Comets and Neveu as used in the standard SK model combined with truncation techniques inspired by a recent paper by Talagrand on the $p$-spin version, we prove that (for $p$ even) the random corrections to the free energy are on a scale $N^-(p-2)/4$ only, and after proper rescaling converge to a standard Gaussian random variable. This is shown to hold for all values of the inverse temperature, $b$, smaller than a critical $b_p$. We also show that $b_prightarrow sqrt2ln 2$ as $puparrow +infty$. Additionally we study the formal $puparrow +infty$ limit of these models, the random energy model. Here we compute the precise limit theorem for the partition function at it all temperatures. For $b< sqrt2ln2$, fluctuations are found at an it exponentially small scale, with two distinct limit laws above and below a second critical value $sqrtln 2/2$: For $b$ up to that value the rescaled fluctuations are Gaussian, while below that there are non-Gaussian fluctuations driven by the Poisson process of the extreme values of the random energies. For $b$ larger than the critical $sqrt2ln 2$, the fluctuations of the logarithm of the partition function are on scale one and are expressed in terms of the Poisson process of extremes. At the critical temperature, the partition function divided by its expectation converges to $1/2$.