Abstract Solar tower power plants rely on precise calibrations of their heliostats for efficient operation. Open-loop calibration procedures are the most common type due to their cost-effectiveness. Two main approaches to these algorithms exist: geometry-based robotic kinematics and neural network-based models. While the former is reliable and requires little data, it only yields moderate accuracy. The latter, however, promises higher accuracies but is data-hungry and unreliable. In this study, we propose a 2-layer coarse-to-fine hybrid model that combines the strengths of both approaches. Our model uses a rigid-body model for prealignment, then phases in a neural network disturbance model through a regularization sweep. This approach ensures that the prediction accuracy is, in the worst-case, equivalent to that of the rigid-body model. Moreover, it helps to identify deficiencies that may have been overlooked by the physical approach. It especially is capable to compute deviation from the geometry models averaged optimum. For testing, we used real measurement data from daily heliostat calibration at the solar tower in Jülich. We also employed a training/validation data split for evaluation, which allows for a conservative performance assumption over the entire year. Our results demonstrate that the hybrid-model outperforms rigid-body models starting from the first measurement, achieving a top performance below 0.7 milliradians. In conclusion, our proposed hybrid model provides a cost effective in-situ solution for heliostat calibration with highest accuracies on low data in solar tower power plants for all open loop calibration methods.