The navigation of an autonomous system (robot or vehicle) in an unstructured environment remains an open problem despite the significant progress made in recent years in the field of autonomous vehicles. Recent innovations in the field of perception using Deep-Learning methods suggest solutions that have yet to be put into practice on real applications in complex environments. In the ASTRA (Autonomous System for Terrain Recognition & Adaptation) project, ESIGELEC and SITIA are proposing the development of a system for environmental perception, semantic mapping and navigation adapted to the constraints of difficult real environments: natural, forest or agricultural terrain; unstructured and/or disturbed environments... This system is based on a complete set of sensors (colour and NIR cameras, neuromorphic cameras, LIDAR, RADAR, GNSS-RTK and inertial unit) allowing the acquisition of reliable and rich measurements of the environment. The perceptions from these sensors are then merged by specific algorithms to improve the reliability of the data collected and to compensate for sensor failures or transient defects (occlusions, dust, rain). Finally, a set of higher-level algorithms is based on these merged data to achieve an understanding of the environment (detection of visible or hidden obstacles, qualification of the quality and geometry of the ground); precise localisation that is robust to GNSS failures; and finally, the fully or semi-autonomous navigation of a mobile robot. Man-machine collaboration is not forgotten: several alternative piloting modes are developed, each with a different level of decision-making by the machine, with the objective of a simple and intuitive handover between the human operator and the onboard intelligence. During all phases of the project, the methods and algorithms are developed in a generic and reusable way, so that they can be easily transposed to other types of mobile systems (vehicles, robots of different sizes and with different crossing capacities). This result is guaranteed by the use of 3 robots of different sizes on which the algorithms are tested and validated at each stage of the challenge. The new methods developed during this project will be valorised by a technological transfer to the field of autonomous agricultural robotics, of which SITIA is one of the leaders.