Manipulating everyday objects without detailed prior models is still beyond the capabilities of existing robots. This is due to many challenges posed by diverse types of objects: Manipulation requires understanding and accurate model of physical properties of objects such as shape, mass, friction, elasticity, etc. Many objects are deformable, articulated, or even organic with undefined shape (e.g., plants) such that a fixed model is insufficient. On top of this, objects may be difficult to perceive, typically because of cluttered scenarios, or complex lighting and reflectance properties such as specularity or partial transparency. Creating such rich representations of objects is beyond current datasets and benchmarking practices used for grasping and manipulation. In this project we will develop an automated interactive perception pipeline for building such rich digitization. More specifically, in IPALM, we will develop methods for the automatic digitization of objects and their physical properties by exploratory manipulations. These methods will be used to build a large collection of object models required for realistic grasping and manipulation experiments in robotics. Household objects such as tools, kitchenware, clothes, and food items are not only widely accessible and in focus of many practical applications but also pose great challenges for robot object perception and manipulation in realistic scenarios. We propose to advance the state of the art by including household objects that can be deformable, articulated, interactive, specular or transparent, as well as shapeless such as cloth and food items. Our methods will learn physical properties essential for perception and grasping simultaneously from different modalities: vision, touch, audio as well as text documents such as online manuals and will include the following properties: 3D model, texture, elasticity, friction, weight, size and grasping techniques for intended use. At the core of our approach is a two-level modeling, where a category level model provides priors for capturing instance level attributes of specific objects. We will exploit online available resources to build prior category level models and a perception-action-learning loop will use the robot’s vision, audio, and touch to model instance level object properties. In return, knowledge acquired from a new instance will be used to improve the category-level knowledge. Our approach will allow us to efficiently create a large database of models for objects of diverse types, which will be suitable for example for training neural network based methods or enhancing existing simulators. We will propose a benchmark and evaluation metrics for object grasping, to enable comparisons of results generated with various robotics platforms on our database. The main objectives we pursue are commercially relevant robotics technologies, as endorsed by the support letters of several companies. We will pursue our goals with a consortium that brings together 5 world-class academic institutions from 5 EU countries (Imperial College London (UK), University of Bordeaux (France), Institut de Robòtica i informàtica Industrial (Spain), Aalto University (Finland), and the Czech Technical University (Czech Republic), assembling a complementary research team with strong expertise in the acquisition, processing and learning of multimodal information with applications in robotics.

"Our higher education institutions represent more than 180,000 students and 50,000 graduates each year, 16,000 faculty members and 11,000 administrative staff. Following the motto ""United in diversity"", EELISA brings together complementary strengths and profiles across Europe with respect to engineering education and political background. We envision a future in which society thrives and masters global challenges with smart and sustainable solutions empowered by European engineering. The fundaments for this future must be laid in higher education. Thus, we want to transform European engineering education in a way that we open learning up and combine innovative teaching methods across our institutions to convey basic knowledge more efficiently. This will be significantly rounded out with interdisciplinary learning, the development of transferable skills and real-world problem solving together with extra-university partners. EELISA's mission is ultimately to transform the whole European Higher Education Area, converting it into a central actor and a model in Europe for solving societal challenges and empowering student citizenship participation and employability by: Re-inventing the ""European engineer""; Democratizing engineering education; Evolving interdisciplinary engineering learning; Fostering knowledge and technology transfer; Stimulating inclusiveness; inspiring others. The prototype of our ambition will be ""European engineering"", which is consistent with our main area of activity. We regard this as a good proof-of-concept for the whole education sector and plan to build this prototype based on two pilot tracks, directly related to Sustainable Development Goals: Smart, Green and Resilient Cities and 2) Sustainable and Smart Industry. In the immediate years, we will follow up other priorities of the Horizon Europe agenda. Our mission implementation will be accelerated by our three strategic mission boosters: communities, diplomas and credentials, and campuses"

Solidification of metallic alloys leads to the formation of complex microstructures which influence greatly their behaviour during subsequent transformations or their service properties and frequently to the formation of defects like pores or hot tears which are very detrimental for these properties. It happens sometimes that hot tears lead to the rejection of the solidified products. Numerous coupled phenomena are at the origin of these defects such as the deformation of the solid network subjected to external stresses, solidification shrinkage, interdendritic liquid flow. These phenomena are greatly influenced by the microstructure resulting from solidification which depends itself on the solidification rate, on the addition of refining elements or elements which can modify the solid-liquid interfacial energy. In order to better understand the influence of all these phenomena, it is necessary to develop numerical simulations at the scale of a representative elementary volume (REV) of the microstructure. These simulations must be supplied with physical data of the material and validated by appropriate experiments and pertinent observations. Some years ago, these observations were carried out after complete solidification only and on metallographic sections. X-Ray tomography allowed recently performing 3D precise characterisations and in addition in-situ owing to the development f fast acquisition cameras. Therefore, it is now possible to get images of an alloy during its solidification and developments are under way to visualise in 3D and in-situ the formation of defects when an alloy is strained at a given temperature in the mushy zone (constrained solidification or solidification with imposed deformation). The objective of this project is to use in-situ X-Ray tomography to obtain input data for the numerical simulation of the phenomena occurring during solidification and to validate the results of the simulation by in-situ X-ray tomography. In addition to tomography, mechanical tests will be carried out to characterise the behaviour of the solid phase at the temperatures corresponding to the solidification range. The solidification structures deduced from microtomography will be used to obtain a spatial discretisation of the REV. A 3D-modelling of the level set type will be developed in order to model the evolution of the microstructure during the experiment. This evolution will be calculated as a function of the thermal history imposed at the REV surface. This information could be obtained from macroscopic simulations of solidification carried out with the THERCAST computer code. In a second step, mechanical conditions will be imposed at the surface of the REV: a velocity field for the solid phase and a pressure gradient for the liquid phase, again calculated by a macroscopic model. The challenge will then to model the evolution of the REV subjected to these complex conditions. The variation of the fraction of the phases and the morphology of the microstructure will be followed. The aim is to understand how such a solid-liquid structure deforms: how do the grains deform during their growth, how do the liquid flow in between the interdendritic spaces, is it possible to predict the formation of pores and cracks in the liquid films, is it possible to model the development of an inter dendritic tearing? The third part of the project is concerned with the validation of the modelling by in-situ experiments during solidification and the prediction of the influence of the various involved parameters. Three aspects will be covered: free solidification which will allow comparing real microstructures with those resulting from the simulation, solidification with shear deformation which will allow validating simulations concerning rheological behaviour of the mushy alloy at various solid fractions, finally solidification under tension which will allow validating simulations of hot tear formation. For these last experiments involving deformation, local strain fields will be analysed by 3D image correlations and compared with results of the numerical simulations. This project will therefore lead to a much better understanding of the rheological behaviour of alloys in their mushy state and of the formation of defects. The influence of the main parameters which are important for solidification should be better understood at the end of this collaborative project which associates experimentalists and modellers.