Stage M2 - Shape Optimal Combination of Multiple Segmentations and Probability Maps in Medical Imaging

M2 Internship Opportunity – CREATIS Laboratory (Villeurbanne, Lyon)

Title : Shape Optimal Combination of Multiple Segmentations and Probability Maps in Medical Imaging

Supervisors : 

Stéphanie Jehan-Besson (Chercheur CNRS, Laboratoire CREATIS), Patrick Clarysse (Directeur de Recherche CNRS, Laboratoire CREATIS), in collaboration with Stefan Duffner (Professeur INSA de Lyon, LIRIS)

Contact: stephanie.jehan-besson@cnrs.fr

Team: https://creatis-myriad.github.io , specialized in methodological approaches for medical image processing, with a strong focus on deep learning.

Workplace : Laboratory CREATIS, Center for Research in Image Acquisition and Processing for Health, UMR CNRS 5220, INSERM U1294, Université Lyon 1, INSA Lyon (https://www.creatis.insa-lyon.fr/) 

Context and Objective 

In the field of 2D or 3D region of interest (ROI) delineation in medical imaging, combining segmentations of anatomical structures from different sources is of great interest, particularly with the growing use of multi-modal and multi-parametric imaging devices. Furthermore, merging several expert delineations of the same ROI can help estimate a consensus ground truth, accounting for intra- and inter-expert variability. Evaluating this variability and combining multiple segmentation methods or expert annotations to obtain a reference shape is therefore highly relevant. 

It is also important to consider combining results from different segmentation methods, or from the same method applied with varying parameters, initializations, training datasets, or architectures. Such an approach can eliminate the need to select a single method with fixed settings, thereby improving reproducibility.

The combination of segmentations enables the computation of a reference shape, which can be considered as the « true » segmentation, derived from multiple « observations » provided by experts or algorithms. This concept was first introduced by the pioneering STAPLE algorithm [1]. The reference shape can also be considered as a mean shape or a barycenter of shapes through the optimization of various criteria. Our goal is to optimally combine several segmentation results, which may sometimes contain partial information. So, in certain cases, a simple mean shape does not provide an optimal consensus but only an average estimate.

The originality of our work consists in the estimation of what we call a « mutual shape », obtained through an optimization criterion that may involve various shape metrics, including an original metric based on information theory [2]. This metric has proven effective for robust fusion of 2D and 3D segmentation methods [2,3,4] and may be defined as a an estimate of a probabilistic measure of a symmetric difference. The optimal shape is obtained through the deformation of an initial contour, or surface guided by shape gradients computed using shape optimization tools [2,3,4]. The active shape is implemented using the well known level set method in the Pandore library or the ITK library in C++.

So far, the segmentations we have used are represented by binary masks. The objective of this internship is to extend this work to the fusion of probability maps, in order to better account for uncertainty in the data. For example, combining probability maps produced by deep learning segmentation methods can avoid the thresholding step before merging, thus preserving all the information contained in these maps. This approach requires distances between probability distributions rather than distances between shapes. Consequently, we aim to adapt our information-theoretic metric to handle probability maps and compare its performance with other distances between probability densities. This part of the work involves designing a new continuous model for the information-theoretic criterion.

Internship Structure

Part 1  - Deep Learning Segmentation in Medical Imaging
The first part will focus on studying classical deep learning algorithms for segmentation in medical imaging. We will concentrate on the segmentation of cardiac structures in echocardiography, using open-source tools developed by the MYRIAD team and the CAMUS database, which was created by MYRIAD and is widely used by the international community.
This initial step will involve generating probability maps that exhibit variability, enabling the study of suitable metrics for probability maps fusion. The student will also conduct a literature review on fusion methods used to combine probability maps produced by deep learning approaches (see, for example, [5]).

Part 2  - Fusion of probability maps and Mutual Shape Estimation
In the second part of the internship, the established datasets will be analyzed through the lens of segmentation combination via mutual shape estimation. The method we are developing is based on shape optimization and shape gradient computation.
The goal is to adapt these approaches to account for uncertainty provided by probability maps, thus extending mutual shape estimation to a probabilistic framework. Additionally, other distances between probability densities may be explored for comparison.

Long-Term Perspective (SMIP Project in collaboration with LIRIS within the FIL Federation of Computer Science of Lyon)

In the longer term, the objective is to investigate how metric learning algorithms [6,7] can be integrated to compute shape barycenters or mutual shapes. Projecting probability maps into an adapted space using learning-based methods could help linearize computations when the estimation becomes too complex. This may also facilitate the comparison of different probability-based distances for mutual shape estimation.
Indeed, optimizing certain distances requires complex derivations to compute shape gradients, which is not always feasible. A reflection on these methods could be considered during the internship in collaboration with S. Duffner.

Required Skills

  • Strong background in image processing or computer vision. 
  • Knowledge of basic deep learning techniques
  • Familiarity with probability theory and optimization methods
  • Programming skills (Python; experience with medical imaging libraries such as ITK is a plus). 

Starting date: March or April 2026

  • Duration : 17-20 weeks

Bibliography:

  • [1] «Simultaneous truth and performance level estimation (STAPLE) : an algorithm for the validation of image segmentation» S.K.Warfield, K. H. Zou et W. M. Wells III In : IEEE TMI (2004)
  • [2] « A Mutual Reference Shape for Segmentation Fusion and Evaluation », S. Jehan-Besson, R. Clouard, C. Tilmant, A. de Cesare, A. Lalande, J. Lebenberg, P. Clarysse, L. Sarry, F. Frouin, M. Garreau, arXiv:2102.08939 (2021).
  • [3] « Optimization of a mutual shape based on the Fréchet-Nikodym metric for 3D shapes fusion », S.Jehan-Besson, P. Clarysse, R. Clouard, F. Frouin.  International Conference on Curves and Surfaces (2022).
  • [4] « Optimization of a shape metric based on information theory applied to segmentation fusion and evaluation in multimodal MRI for DIPG tumor analysis », S. Jehan-Besson, R. Clouard, N. Boddaert, J. Grill, F. Frouin, Int. Conf . on Geometric Science of Information GSI (2021).
  • [5] « Deep Model Fusion: A Survey », W. Li, Y. Peng, M. Zhang, L. Ding, H. Hu, L. Shen, arXiv :2309.15698 (2023)
  • [6] « Learning Wasserstein Embeddings », N. Courty, R. Flamary, M. Ducoffe, ICLR (2018)
  • [7] « Similarity Metric Learning », S. Duffner, C. Garcia, K. Idrissi , A. Baskurt, Book Chapter In Multi-faceted Deep Learning - Models and Data, Springer, (2021)