Abstract Thermoelectric generator (TEG) has attracted considerable attention for the waste heat recovery of internal combustion engine. In this study, a 3-D numerical model for engine exhaust-based thermoelectric generator (ETEG) system is developed. By considering the detailed geometry of thermoelectric generator (TEG) and exhaust channel, the various transport phenomena are investigated, and design optimization suggestions are given. It is found that the exhaust channel size needs to be moderate to balance the heat transfer to TEG modules and pressure drop along channel. Increasing the number of exhaust channels may improve the performance, however, since more space and TEG modules are needed, the system size and cost need to be considered as well. Although only placing bafflers at the channel inlet could increase the heat transfer coefficient for the whole channel, the near wall temperature downstream might decrease significantly, leading to performance degradation of the TEG modules downstream. To ensure effective utilization of hot exhaust gas, the baffler angle needs to be sufficiently large, especially for the downstream locations. Since larger baffler angles increase the pressure drop significantly, it is suggested that variable baffler angles, with the angle increasing along the flow direction, might be a middle course for balancing the heat transfer and pressure drop. A single ETEG design may not be suitable to all the engine operating conditions, and making the number of exhaust channels and baffler angle adjustable according to different engine operating conditions might improve the performance.