Abstract This paper focused on the intake and exhaust structure optimization of a single-screw expander (SSE) by using numerical method. The effects of internal volume ratio, the shape and clearance height of the intake port and exhaust area were numerically analyzed (R123 and HFO-1336mzz(Z) were selected as working fluids) to reduce the energy losses (more than 30%) caused by pressure loss and leakage in intake process and exhaust pressure loss. The suggestions of reducing the internal volume ratio and clearance height at intake port and removing the closed helix line were given based on the simulation results. Then, the optimized SSE was developed and experimentally tested in ORC system with R123 as working fluid. Results showed that the filling factor of the prototype was reduced from 125% to nearly 100% and the highest shaft efficiency was increased from 56% to 67.7% at 3000 rpm. The enhanced performance of the new prototype proves the correctness of the numerical optimization and this is expected to provide guidance for the design of SSE.