Title: Deep learning segmentation of premature neonate cerebral structures in 3D ultrasound imaging 

Research Unit: CREATIS

PhD thesis supervisor: Philippe Delachartre – Creatis – INSA Lyon

Important dates

The deadline for submission is April 30th at 5pm. The results will be released before May 15th.

The PhD thesis will start at the beginning of October.

Context

The clinical practice in the field of preterm newborn follow-up is based on visual inspection of 2D ultrasound brain imaging. It is the first line of testing. It takes place at the patient's bedside, it is repeatable for medical follow-up (3 to 5 ultrasound exams per premature infant) and it is not expensive. It can be used for all preterm newborns. In case where abnormality is suspected, a 3D MRI examination is done. This concerns 15 % of the preterm newborn population. Several limitations can be pointed out in this process. (i) The Epipage study [Ghada-10] showed that some infants were not selected for MRI and will have brain abnormalities (irreversible motor disorder due to brain lesions or abnormal development of brain structures), (ii) The MRI contrast is lower in preterm newborn brain structures compared to children or adults, (iii) MRI is not adapted to a patient follow-up.

Key words: machine and deep learning, segmentation, medical imaging

Aim of the thesis

The goal of this PhD project is to develop methods and tools (i) to reconstruct 3D ultrasound data, (ii) to segment specific brain structures as cerebellum (a biomarker of cerebral development) and brain stem, (iii) to segment the corticospinal tract, (iv) to quantify the structures in volume for the cerebellum and the brain stem, (v) to quantify and to locate the disconnection in the fiber beam to later propose a prognostic model and to improve patient management.

Scientific challenges 

The current success of machine learning methods in real-world applications, especially the impressive results obtained using deep learning, largely depends on the size of the annotated sample available for learning. In medical imaging, the problem of acquiring sufficiently big data sets is widely present and is often due to ethical issues as well as time consuming processes required to build annotated databases. Moreover, acquisition and processing of 3D brain ultrasound data stills are challenging due to the speckle nature of ultrasound and a limited field of view. Therefore, finding solutions for data acquisition and reconstruction, weakly supervised segmentation and data augmentation with GAN present major challenges in ultrasound imaging.

Expected original contributions 

We expect to produce new efficient algorithms for data reconstruction, segmentation of cerebellum and tracts in 3d ultrasound imaging and to answer to the question: how to avoid time-consuming manual labeling of samples by using already available manually segmented images?

Research Program   

This thesis proposal is structured around three different axes which are data reconstruction, segmentation, data simulation (augmentation) for clinical application.

- Data reconstruction

At the beginning of the project, the images will be acquired, plane by plane, by moving the ultrasound probe inside the fontanelle. The advantage is that we can get high quality images using a pseudo-matrix probe. The drawback is that we need to develop a geometric model to convert the dynamic data sequence into a reconstructed volume. The limited field of view in ultrasound imaging requires the physician to perform dedicated acquisition depending on the targeted structures: one for cerebellum and another one for brain stem and corticospinal tract. Therefore the geometric acquisition model should be adjusted for both cases. Towards the end of the project we should be able to capture the 3D data directly from a 2D matrix array provided that the image quality and the field of view will be high enough.

- Data segmentation

A preliminary study showed us that our active contour methods [Sciolla-17] and models [Farouj-17] dedicated to 3D ultrasound data is exceeded by the complexity of the structures to segment in brain imaging. A segmentation evaluation has been done on Lateral Ventricles [Sciolla-16] [Martin 18]. To circumvent this problem, we propose using a deep learning approach for segmentation. Several works have used deep learning segmentation for MRI brain imaging, but few works are interested in deep learning segmentation for 3D ultrasound brain data and these works are focused on the lateral ventricles segmentation [Qiu-17] [Martin-18] (figure 1). The multi structure segmentation and 3D aspect of this project require different ways of investigation. Firstly, we propose to investigate the way of weakly supervised algorithms to alleviate the number of manual segmentation in the database and secondly, to take into account the contextual information (shape, location, probabilistic framework) to increase the learning performances of supervised algorithms.

- Data augmentation

Data augmentation has been shown to be an effective method to increase the accuracy of CNN networks in brain MRI [Valverde 17]. Recently, Hu et al. [Hu 17] proposed a GAN architecture, together with the calibrated physical coordinates as conditioning input to reduce the training time and to stabilize the convergence of the network. To complete our database, data augmentation of ultrasound images will be investigated using Generative Adversarial Network (GAN) architectures.

Scientific supervision

The PhD student will be supervised by Philippe Delachartre (Prof, INSA Lyon). The PhD student will benefit from the expertise of Philippe Delachartre in segmentation of ultrasound imaging and that of Dr Philippe Quétin, pediatrician at the Avignon hospital center for data acquisition and analysis.

Use of research results

The results of this project will be presented at top machine learning and medical imaging venues as well as at the dedicated workshops concerning their intersections. The extended experimental evaluations will be submitted to top peer machine learning and medical imaging journals. On the other hand, the developed algorithms will be implemented and made available on dedicated software to be developed allowing users to perform their own clinical research using the algorithms on the collected data sets or on their own database. We expect these new methods to have a strong impact on the work of the scientists involved from the medical imaging community to analyze different kinds of ultrasound data they routinely acquire and that is clearly in demand for quantitative results of cerebral development and prognostic of all preterm newborns.

Pre-requisite qualifications of the applicant

A successful candidate will have strong knowledge in at least two of the following fields: deep learning, applied mathematics, image processing.

Programming skills: Python (TensorFlow, PyTorch).

Skills developed by the successful candidate during the PhD project

The successful candidate will develop strong skills in deep learning, medical imaging analysis, data simulation and medical application. Due to the theoretical and applicational aspects of this proposal, the successful PhD student will be able to join research departments in both industry and academia.

References 

[Farouj-17] Y. Farouj, Freyermuth, J. M., Navarro, L., Clausel, M., Delachartre, P., « Hyperbolic Wavelet-Fisz denoising for a model arising in ultrasound imaging », IEEE Transactions on Computational Imaging, vol. 3(1), p. 1-10, 2017. DOI: 10.1109/TCI.2016.2625740.

[Ghada-10] B. Ghada, et al. Predictors of Cerebral Palsy in Very Preterm Infants: The EPIPAGE Prospective Population-Based Cohort Study. Developmental Medicine and Child Neurology 52(6): 119-125, 2010.

[Hu 17] Hu Y. et al. Freehand Ultrasound Image Simulation with Spatially-Conditioned GANs. In: Cardoso M. et al. (eds) Molecular Imaging. Lecture Notes in Computer Science, vol 10555. Springer, 2017.

[Martin-18] M. Martin, B. Sciolla, P. Quetin, P. Delachartre, Automatic segmentation of the cerebral ventricle in neonates using deep learning with 3D reconstructed freehand ultrasound imaging, IEEE International Ultrasonics Symposium (IUS), Kobe, pp. 1-4, 2018.

[Qiu-17] Qiu, Wu, et al.: Automatic segmentation approach to extracting neonatal cerebral ventricles from 3D ultrasound images. Medical image analysis 35: 181-191, 2017.

[Sciolla-16] B. Sciolla, M. Martin, P. Delachartre, P. Quetin, "Segmentation of the lateral ventricles in 3D ultrasound images of the brain in neonates," IEEE International Ultrasonics Symposium (IUS), Tours, pp. 1-4, 2016. DOI: 10.1109/ULTSYM.2016.7728560.

[Sciolla-17] B. Sciolla, Cowell, L., Dambry, T., Guibert, B., Delachartre, P., « Segmentation of Skin Tumors in High-Frequency 3D Ultrasound Images », Ultrasound in Medecine and Biology, vol. 43, p. 227–238, 2017.

[Valverde 17] Valverde, Sergi et al. “Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach.” NeuroImage 155: 159-168, 2017.