The increasing quantities of data generated by modern industrial and business processes pose enormous challenges for organizations seeking to glean knowledge and understanding from the data. Combinations of HPC, Cloud and Big Data technologies are key to meeting the increasingly diverse needs of large and small organizations alike. Critically, access to powerful compute platforms for SMEs - which has been difficult due to both technical and financial reasons - may now be possible. LEXIS (Large-scale EXecution for Industry & Society) project will build an advanced engineering platform at the confluence of HPC, Cloud and Big Data which will leverage large-scale geographically-distributed resources from existing HPC infrastructure, employ Big Data analytics solutions and augment them with Cloud services. Driven by the requirements of the pilots, the LEXIS platform will build on best of breed data management solutions (EUDAT) and advanced, distributed orchestration solutions (TOSCA), augmenting them with new, efficient hardware capabilities in the form of Data Nodes and federation, usage monitoring and accounting/billing supports to realize an innovative solution. The consortium will develop a demonstrator with a significant Open Source dimension including validation, test and documentation. It will be validated in the pilots - in the industrial and scientific sectors (Aeronautics, Earthquake and Tsunami, Weather and Climate) – where significant improvements in KPIs including job execution time and solution accuracy are anticipated. LEXIS will promote the solution to the HPC, Cloud and Big Data sectors maximizing impact through targeted and qualified communications. LEXIS brings together a consortium with the skills and experience to deliver a complex multi-faceted project, spanning a range of complex technologies across seven European countries, including large industry, flagship HPC centres, industrial and scientific compute pilot users, technology providers and SMEs.

EXA4MIND will build an Extreme Data platform that brings together large scale data storage systems and powerful computing infrastructures by implementing novel automated data management and effective data staging. This will enable integration with extreme data sources and support complex and advanced data analysis pipelines for knowledge extraction. Involving three large universities and a research centre, small and medium-sized enterprises (SMEs), and the industrial sector, EXA4MIND uses interdisciplinary top-level expertise to create a modular Extreme Data Database (EDD) analytics environment. Driven by four representative Extreme Data Application Cases from the fields of molecular dynamics, autonomous driving, smart agri-/viticulture and health/social Big Data, EXA4MIND will support data ingestion and versatile preprocessing, develop an “Advanced Query and Indexing System” with convenient interfaces, and leverage high-performance storage and computing infrastructure. We will make Big Data from heterogeneous storage systems and database management systems available in a uniform way for processing on European pre-Exascale Supercomputers, cloud and AI-processing systems. Further to providing automatised staging, caching or direct access to these systems, the results and relevant data from our project will be made available to the European FAIR Data Ecosystem (including European Data Spaces, EUDAT, and the EOSC). Therefore, standardised modules and automated procedures will be implemented in our EDD. The project will constitute a leap forward towards common European Big and Extreme Data techniques in science, industry, and SMEs. Offering advanced data workflow orchestration, the EDD will be validated, proven, and sustained in collaboration with application partners, generating internationally recognised results. EXA4MIND will thus lay the technological foundations for future European data analytics and computation taking into account the whole Digital Continuum.