MesoNet – Juliet and Jupyter

How to launch JupyterLab on Juliet?

The MesoNet cloud provides CPU/GPU resources and storage.

Jupyterlab is a notebook server used for practical sessions.

The road is long to make them work together. You need to follow all these steps :

Create an account
Set up a secure SSH connection
Log on to the Juliet server
Ask for a resource on the Juliet server
Launch a JupyterLab server on the obtained resource
Create an SSH tunnel to the JupyterLab service
Connect locally to JupyterLab

Remark: local means your computer. server means distant resources.

Mesonet Account

Read the following documentation: https://www.mesonet.fr/documentation/user-documentation/acces/portail

As EduGAIN is available, French students, teachers, researchers, and engineers can access it through their academic account.

SSH connection setup

The documentation (in french) is here. These steps are required and are done only one time.

3 steps are needed :

Create your SSH key. On your local computer, open a terminal (Windows/Linux) and use the following command to create a new SSH key. You will also be asked to enter a passphrase to encrypt your private key. Don’t lose this passphrase. Never send or communicate your private key! Only the public one (ending in .pub).
(local)$ ssh-keygen -t ed25519 -f .ssh/id_ed25519
Put your public key on Mesonet (refer to this documentation)
To add your SSH key, log in to the MesoNET portal at https://acces.mesonet.fr. Click the Log In button and sign in the same way you did when creating your account—typically via eduGAIN. Once authenticated, click the User icon in the top-right corner, then select SSH Keys. This opens the SSH key management page.
Associate your public key with a project (read this page)

Log in to Juliet

Use an SSH connection to juliet.mesonet.fr with the username given by email or on the website (ie tgrenier). See here for more details. As an example, here is the command to enter on your computer (local) – change USERNAME to your real login:

(local)$ ssh USERNAME@juliet.mesonet.fr

Obtain resources on Juliet

You have to submit a job on Juliet to obtain access to a GPU. For our particular usage with JupyterLab and as a “beginner”, we will submit an interactive job with the following commands.

1- Obtain the project ID. You will get a value such as: m2XXXX

(juliet1)$ sacctmgr --noheader --parsable2 show association where user=$USER format=account

2- Launch an interactive job. Change the m2XXXX to the value obtained with the previous command. This command asks for a 4-hour interactive job, 4 CPUs, and 1 GPU.

(juliet1)$ srun --time=4:00:00 --account=m2XXXX --pty -c 4 --gres=gpu:1 /bin/bash

When the srun command succeeds and a resource is obtained, you are logged on the computational node with a new bash. In the following example, from juliet1 (the server), I obtain an interactive job on juliet2. It is important to note the name of this NODE (here juliet2).

Launch a JupyterLab server

If not already done, you have to install Miniconda.

(julietN)$ cp /projects/m26238/Install/Miniconda3-latest-Linux-x86_64.sh ~/
(julietN)$ bash ~/Miniconda3-latest-Linux-x86_64.sh

Activate the conda environment used for this practical session (how to create it yourself is described here):

(julietN)$ conda activate /projects/m26238/Keras3TF

Launch JupyterLab with:

(julietN)$ jupyter lab

Have a look at the output of this command. It will give you i) the PORT number (8888 here) and ii) the TOKEN (a69ab…. ).

Create an SSH tunnel to the JupyterLab server

In a new local terminal, adjust the following command to your USERNAME, your NODE, the PORT of the JupyterLab server, and your USERNAME

(local)$ ssh -J USERNAME@juliet.mesonet.fr -L 8000:localhost:PORT USERNAME@NODE

It can look like:

(local)$ ssh -J tgrenier@juliet.mesonet.fr -L 8000:localhost:8888 tgrenier@juliet2

Access to JupyterLab

In your favorite web browser (from your local computer), open the address 127.0.0.1:8000. When asked for a password or token, use the TOKEN of the JupyterLab and log in.

Enjoy!

Now you can download the material on classification here, or segmentation here, or… and drag and drop this file into the left panel.