During the past few years, there has been enormous progress in the fields of Deep Learning and Artificial Intelligence (AI), with huge potential for use in medicine and industry. To deepen knowledge among its Eastern Partnership (EaP) partners, the EaPConnect project organised two workshops, on Data Mining / Deep Learning and on Containers in High-Performance Computing (HPC). The workshops were part of an award by the Enlighten Your Research Eastern Partnership programme to distribute knowledge about AI and machine learning. The workshops were held in conjunction with HPC Summit week, in Ljubljana on 28 May to 1 June, to allow discussions between experts from EaP and EU countries.
EaP potential for Deep Learning and AI
Deep Learning and AI technologies that are implemented in software and properly trained on large amounts of datasets are crucial to tackle various challenges in industry, medical diagnosis, quality control, marketing, internet services, gaming, the entertainment industry, etc. However, serious obstacles still prevent their widespread use:
- To ‘teach’ artificial deep neural networks a very large amount of data is required, which is difficult to accumulate and label. This is especially true for sensitive areas such as medical imaging and computerised diagnosis.
- Training of ‘Cs’ (CNN) is a large-scale optimisation task, which is computationally very expensive.
In order to remove these obstacles, we want to share our knowledge and medical image data with European partners, to achieve better scientific results, develop Deep Learning and Data Mining technologies further, and contribute towards better health services in EU and EaP countries.”
There is a lot of potential for AI development and application in the EaP region, but it will unfold differently in the various countries. Data is critical to the successful application of AI and data concentration may vary, meaning that AI will develop at different rates. Nevertheless, there is an enormous pool of data to work through, so AI adoption may progress only slowly.
One factor that could set the EaP region’s AI development apart is the relative scarcity of traditional data in many markets. This means that alternative and unstructured data will play an bigger role in AI implementation. The ability to analyse large amounts of alternative data will take things a step further. AI is expected to unlock the potential of alternative data sets, which originate as a product of a company’s operations, opening the door to many new products and services.In EaP countries, participants are working on AI and Deep Learning technology applications in computer-assisted medical diagnosis, new materials development, computational physics, environmental protection and other areas. They are very interested in information on accessing computational resources for long-lasting training of deep CNNs and related information resources. The main hardware characteristic is the availability of recent high-performance computational resources including using modern accelerators like GPU NVidia graphical cards. Another critical point is creating joint, open-access information resources and crowd sourcing.
The use of patients’ image data will continue to be a very sensitive and ethically/security critical issue, even when data are completely de-personalised. Thus, another topic for consideration is possible ways of using brand-new technologies such as Generative Adversarial Networks. After training two competitive networks, this technology may be able to generate artificial medical images that are indistinguishable from real patient images. If so, this could open a new era of free use of medical images of ‘people who do not really exist’ in training algorithms, handbook illustrations, teaching, websites and other resources.
Containers in HPC workshop
The ‘Containers in HPC’ workshop gave an overview of Linux containers technology, Docker installation, building and managing Docker containers, building singularity containers, converting Docker containers to singularity, and running singularity containers on the HPC cluster. Participants also completed a Singularity MPI/GPU practical exercise.
This workshop allowed me to gather a lot of current information on HPC computing and HPC cloud computing, both as a HPC researcher and user, and as an engineer and administrator of the MSU HPC Cluster. I look forward to implementing on our HPC cluster what I learned about Docker and Singularity, which allow great benefits for the HPC users and me as a HPC administrator, especially software packaging and porting to any Linux environment.”
The event provided a great opportunity for me to stay up to date with the latest trends in the industry. The networking proved to be invaluable, as the long term relationships have a high chance to become very fruitful. My special thanks to Abdulrahman Azab Mohamed for this workshop, which answered some crucial questions that will help with my further research.”