In this article, a novel embedded low-cost system for real-time fault diagnosis of photovoltaic (PV) modules is proposed. The idea aims to develop an embedded application to classify certain defects that can frequently occur on PV modules based on infrared (IR) images in different regions (desert and Mediterranean climates). The investigated faults are sand accumulation, dirt on PV modules, degradation, and junction box overheating. After several inspections, these are the most commonly observed defects on PV modules in both regions (south and north of Algeria). A tiny convolutional neural network (TinyCNN) was developed, optimized, and integrated into a low-cost and low-power microcontroller (Arduino Nano 33 BLE sense). In this regard, a database of IR thermography images was built and used. The developed TinyCNN-based model could be run locally, without the need to send the data to the cloud for analysis and processing. Another microcontroller [Arduino Nano 33 Internet of Things (IoT)] was used to remotely monitor the state of the PV modules. Thanks to IoT technology, the results have been visualized and posted online on a dedicated monitoring webpage. The proposed embedded solution could be integrated into an unmanned aerial vehicle for real-time applications. Furthermore, it assists operators in diagnosing their PV modules and making a maintenance schedule. The proposed technique outperforms the existing solutions in terms of cost, consuming power, simplicity, and execution time. Simulation and experimental results clearly report the feasibility of the proposed embedded system, which has an average cost of around 120 US dollars.