We investigated the evolution of the spectrally resolved absorption coefficients of SiC and SiOx materials as well as of their multilayer systems during thermal annealing and hydrogen passivation, with focus on the nature of optically active defects induced during annealing. We propose that both dangling bonds (paramagnetic defects) and strained bonds (non-paramagnetic defects) formed during annealing contribute to the sub-band gap absorption and that the associated defects can be partially removed by hydrogen reincorporation. The difference in the evolution of the absorption spectra for different sample types upon annealing and passivation are linked to the fundamental difference in their atomic structures. The much lower optical band gap and the significantly higher sub-band gap absorption of SiC single layers in the annealed state as compared to SiOx single layers can be traced back to the lower flexibility of the relatively dense 4-fold coordinated atomic structure of the SiC material.