Evaluation procedure

Goals
The aims of the evaluation metrics involved in the segmentation contest are twofold:
 measure the degree of accuracy of the left ventricular endocardium and epicardium as the left atrium. This will be done through global and local measures of similarity with the reference contours;
 measure the degree of accuracy of the derived clinical indices.
Clinical metrics
The clinical metrics are the ones that are the most widely used in cardiac clinical practice:
 Left ventricular cavity
 Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
 Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
 Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
 Bias computed from the set of EDV measurements
 Bias computed from the set of ESV measurements
 Bias computed from the set of EF measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
Geometrical metrics
A set of geometrical metrics are computed per structure, i.e. left ventricle endocardium (LV endo), left ventricle endocardium (LV epi) and left atrium (LA), to allow the computation of ranking per structure. The following metrics are computed:
 LV endo
 The average Dice value for the left ventricle cavity at ED.
 The average Dice value for the left ventricle cavity at ES.
 The average mean absolute distance for the endocardial contour of the left ventricle at ED.
 The average mean absolute distance for the endocardial contour of the left ventricle at ES.
 The average Hausdorff distance for the endocardial contour of the left ventricle at ED.
 The average Hausdorff distance for the endocardial contour of the left ventricle at ES.
 LV epi
 The average Dice value for the area composed by the myocardium and the left ventricle cavity at ED.
 The average Dice value for the area composed by the myocardium and the left ventricle cavity at ES.
 The average mean absolute distance for the epicardial contour of the left ventricle at ED.
 The average mean absolute distance for the epicardial contour of the left ventricle at ES.
 The average Hausdorff distance for the epicardial contour of the right ventricle at ED.
 The average Hausdorff distance for the epicardial contour of the right ventricle at ES.
 LA
 The average Dice value for the left atrium region at ED.
 The average Dice value for the left atrium region at ES.
 The average mean absolute distance for the left atrium contours at ES.
 The average mean absolute distance for the left atrium contours at ED.
 The average Hausdorff distance for the left atrium contours at ED.
 The average Hausdorff distance for the left atrium contours at ES.
 measure the degree of accuracy of the left ventricular endocardium and epicardium as the left atrium. This will be done through global and local measures of similarity with the reference contours;
This work has published to IEEE TMI journal. You must cite this paper for any use of the CAMUS database
S. Leclerc, E. Smistad, J. Pedrosa, A. Ostvik, et al.
"Deep Learning for Segmentation using an Open LargeScale Dataset in 2D Echocardiography" in IEEE Transactions on Medical Imaging, vol. 38, no. 9, pp. 21982210, Sept. 2019.
doi: 10.1109/TMI.2019.2900516