Context
Recently, neuronal radiance fields (NERFs, [1]) have provided an elegant solution for 3D scene representation. Their effectiveness has been demonstrated for the synthesis of 2D views.
The original utility of NERFs was the ability to generate new 2D views from a set of 2D images taken from the same scene, and the 3D representation of the scene was not the primary focus. For tomography, the goal is to reconstruct the 3D scene from 2D projections ([2]). Work has been initiated in this direction at CREATIS and LIRIS, and the initial results are promising for conventional X-ray tomography applications.
Electron microscopy is used for studying samples in materials science and biology for cell analysis. Images are taken at various tilt angles using a parallel electron beam passing through the sample. Projections are generally acquired over a total angle of approximately 120-140 degrees, which is much less than 180 degrees. The reconstructed images exhibit missing angle artifacts, and some contours cannot be reconstructed, resulting in elongated appearances of objects ([3]). As the electron beam is destructive, for dose control, the projections are noisy, and the signal-to-noise ratio is low, especially for biological samples (see Figure 1). Contrast is low in thick sample volumes (especially whole cells). Another challenge is the uncertainties in the projection angles and the sample position. Although alignment methods are known [4], they are challenging to apply in the absence of markers and in low-contrast images. Supervised deep learning could provide specific data-driven priors that are more precise than traditional regularization methods.
Figure 1 : Tomographic image of a hydrated cell. Left: projection acquired at an incidence angle of the electron beam. Right: 3D reconstruction of the sample.
Goal
NERFs are an unsupervised learning method that allows for image reconstruction based on the physical model, similar to iterative methods. In this sense, it offers an alternative to iterative algorithms that goes beyond the discrete voxel-based representation of volumes. Several objectives are pursued:
- Test the performance of NERFs in electron microscopy.
- Optimize computation times.
- Compensate for uncertainty in projection angles and sample position.
- Introduce data-driven priors. In the case of cell imaging, the narrow scanning angle could be compensated by considering multiple occurrences of identical cells projected at different angles.
Imaging sensitive samples under the electron beam is of great importance in the field of microscopy because it determines the ability to examine structures of very small size. In cellular imaging, it contributes to advancing our understanding of various conditions. The methods studied in this internship can later be applied to other imaging modalities, particularly in the medical field.
Location
The internship will take place at the CREATIS laboratory, in collaboration with the LIRIS and MATEIS laboratories.
Skills
We are looking for someone with strong computer science skills, particularly in Python programming. Experience in machine learning methods is a plus. It is also expected that the intern has a solid foundation in mathematics and good written and oral communication abilities. They will be working with researchers with different specialties: microscopy, computer science, and tomographic reconstruction.
How to apply
Send : CV, motivation letter, grades
Contact
voichita.maxim@creatis.insa-lyon.fr
sebastien.valette@creatis.insa-lyon.fr
Bibliography
[1] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R Ramamoorthi and Ren Ng, âNeRF: Representing Scenes as Neural Radiance Fields for View Synthesisâ, ECCV 2020, https://www.matthewtancik.com/nerf
[2] L. Shen, J. Pauly and L. Xing, "NeRP: Implicit Neural Representation Learning With Prior Embedding for Sparsely Sampled Image Reconstruction", IEEE Transactions on Neural Networks and Learning Systems, 2022
[3] H. Banjak, T. Grenier, T. Epicier, S. Koneti, L. Roiban, A.-S. Gay, I. Magnin, F. Peyrin, V. Maxim, âEvaluation of noise and blur effects with SIRT-FISTA-TV reconstruction algorithm: Application to fast environmental transmission electron tomographyâ, Ultramicroscopy, Volume 189, 2018, Pages 109-123.
[4] C.O.S. Sorzano, F. de Isidro-Gómez, E. Fernåndez-Giménez, D. Herreros, S. Marco, J.M. Carazo, C. Messaoudi, Improvements on marker-free images alignment for electron tomography, Journal of Structural Biology: X, Volume 4, 2020.