For coal tar hydrocracking process in supercritical gasoline, it requires a new catalyst with high activity under low temperature conditions to improve the light oil yield. The effects of Mo–Co–Y and Mo–Ni–Y series catalyst on the light oil yield were investigated in this paper, and the coke deposition on catalyst and catalyst stability were also investigated. Experimental results show that the Mo–Co–Pd–Y zeolite catalyst with a suitable Co/(Co+Mo) atom ratio and calcination temperature could maximize the light oil yield, and the acid amount of the catalyst is closely related to its activity. Coke deposition on the Mo–Co–Pd–Y zeolite catalyst is very little and the catalyst remains high activity within used time of 8 h.