How To Set Up a JupyterLab Environment on Ubuntu 18.04

The author selected the United Nations Foundation to receive a donation as part of the Write for DOnations program.

Introduction

JupyterLab is a highly feature-rich UI that makes it easy for users, particularly in the fields of Data Science and AI, to perform their tasks. The JupyterLab environments provide a productivity-focused redesign of Jupyter Notebook. It introduces tools such as a built-in HTML viewer and CSV viewer along with features that unify several discrete features of Jupyter Notebooks onto the same screen.

In this tutorial, you’ll install and set up JupyterLab on your Ubuntu 18.04 server. You’ll also be configuring your server to be able to connect to the JupyterLab instance remotely from any web browser, securely, using a domain name.

Prerequisites

In order to complete this tutorial, you’ll need:

• An Ubuntu 18.04 server with a non-root user account with sudo privileges using this Initial Server Setup Guide.
• An installation of the Python Anaconda Distribution on your server. You can use the How To Install the Anaconda Python Distribution on Ubuntu 18.04 tutorial.
• A registered domain name or sub-domain where you’ve access to edit DNS records. This tutorial will use your_domain throughout. You can purchase domains on Namecheap, get a free domain at Freenom, or register a new domain with any registrar of your choice.
• The following DNS records set up for your domain:
  • An A record with your_domain pointing to your server’s public IP address.
  • An A record with www.your_domain pointing to your server’s public IP address. This How to Create, Edit, and Delete DNS Records documentation can help you in setting up these records.

Step 1 — Setting Up Your Password

In this step you’ll set up a password on your JupyterLab installation. It is important to have a password in place since your instance will be publicly accessible.

First, make sure your Anaconda environment is activated. As per the prerequisite tutorial the environment is called base.

To activate the environment, use the following command:

conda activate base

Your prompt will change in the terminal to reflect the default Anaconda environment base:

sammy@your_server:~$

All future commands in this tutorial will be run within the base environment.

With your Anaconda environment activated, you’re ready to set up a password for JupyterLab on your server.

First, let’s generate a configuration file for Jupyter:

jupyter notebook --generate-config

You’ll receive the following output:

Output
Writing default config to: /home/sammy/.jupyter/jupyter_notebook_config.py

Both JupyterLab and Jupyter Notebook share the same configuration file.

Now, use the following command to set a password for accessing your JupyterLab instance remotely:

jupyter notebook password

Jupyter will prompt you to provide a password of your choice:

Output
Enter password:
Verify password:
[NotebookPasswordApp] Wrote hashed password to /home/sammy/.jupyter/jupyter_notebook_config.json

Jupyter stores the password in a hashed format at /home/sammy/.jupyter/jupyter_notebook_config.json. You’ll need this hashed value in the future.

Finally, use the cat command on the file generated by the previous command to view the hashed password:

cat /home/sammy/.jupyter/jupyter_notebook_config.json

You’ll receive an output similar to the following:

{
  "NotebookApp": {
    "password": "sha1:your_hashed_password"
  }
}

Copy the value in the password key of the JSON and store it temporarily.

You’ve set up a password for your JupyterLab instance. In the next step you’ll create a Let’s Encrypt certificate for your server.

Step 2 — Configuring Let’s Encrypt

In this step, you’ll create a Let’s Encrypt certificate for your domain. This will secure your data when you access your JupyterLab environment from your browser.

First, you’ll install Certbot to your server. Begin by adding its repository to the apt sources:

sudo add-apt-repository ppa:certbot/certbot

On executing the command, you’ll be asked to press ENTER to complete adding the PPA:

Output
This is the PPA for packages prepared by Debian Let's Encrypt Team and backported for Ubuntu.

Note: Packages are only provided for currently supported Ubuntu releases.
 More info: https://launchpad.net/~certbot/+archive/ubuntu/certbot
Press [ENTER] to continue or Ctrl-c to cancel adding it.

Press ENTER to continue adding the PPA.

Once the command has finished executing, refresh the apt sources using the apt update command:

sudo apt update

Next, you’ll install Certbot:

sudo apt install certbot

Before you can start running Certbot to generate certificates for your instance, you’ll allow access on port :80 and port :443 of your server, so that Certbot can use these ports to verify your domain name. Port :80 is checked for http requests to the server while port :443 is used for https requests. Certbot shall be making an http request first and then after obtaining the certficates for your server, it will make an https request, which will be proxied through port :443 to the process listening at :80 port. This will verify the success of your certificate installation.

First, allow access to port :80:

sudo ufw allow 80

You will receive the following output:

Output
Rule added
Rule added (v6)

Next, allow access to port :443:

sudo ufw allow 443

Output
Rule added
Rule added (v6)

Finally, run Certbot to generate certificates for your instance using the following command:

sudo certbot certonly --standalone

The standalone flag directs certbot to run a temporary server for the duration of the verfication process.

It will prompt you for your email:

Output
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel): your_email

Enter a working email and press ENTER.

Next, it will ask you to review and agree to the Terms of Service for Certbot and Let’s Encrypt. Review the terms, type A if you accept, and press ENTER:

Output
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: A

It will now prompt you to share your email with the Electronic Frontier Foundation. Type your answer and press ENTER:

Output
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y/N

Finally, you’ll be asked to enter your domain name. Type in your domain name without any protocol specification:

Output
Please enter in your domain name(s) (comma and/or space separated)  (Enter 'c'
to cancel): your_domain
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for your_domain
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/your_domain/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/your_domain/privkey.pem
   Your cert will expire on 2020-09-28. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Certbot will perform domain verification and generate certificates and keys for your domain and store them at /etc/letsencrypt/live/your_domain.

Now that you have set up your Let’s Encrypt certificate, you’ll update your JupyterLab configuration file.

Step 3 — Configuring JupyterLab

In this step, you will edit the JupyterLab configuration to make sure it uses the Let’s Encrypt certificate you generated in Step 2. You will also make it accessible using the password you set up in Step 1.

First, you need to edit the JupyterLab configuration file at /home/sammy/.jupyter/jupyter_notebook_config.py:

nano /home/sammy/.jupyter/jupyter_notebook_config.py

Now, navigate to the line defining the value for c.NotebookApp.certfile and update it as follows:

...
## The full path to an SSL/TLS certificate file.
c.NotebookApp.certfile = '/etc/letsencrypt/live/your_domain/fullchain.pem'
...

Next, find the c.NotebookApp.keyfile variable and set it as shown:

...
## The full path to a private key file for usage with SSL/TLS.
c.NotebookApp.keyfile = '/etc/letsencrypt/live/your_domain/privkey.pem'
...

c.NotebookApp.certfile and c.NotebookApp.keyfile refer to the SSL Certificate, which will be served when you try to access your server remotely using the https protocol.

Next, navigate to the line defining the c.NotebookApp.ip variable and update as follows:

...
## The IP address the notebook server will listen on.
c.NotebookApp.ip = '*'
...

c.NotebookApp.ip defines the IPs that can access JupyterLab running your server. You set it to the * wildcard to allow access from any computer you need to access JupyterLab on.

Next, find the c.NotebookApp.open_browser configuration and update it as follows:

...
## Whether to open in a browser after starting. The specific browser used is
#  platform dependent and determined by the python standard library `webbrowser`
#  module, unless it is overridden using the --browser (NotebookApp.browser)
#  configuration option.
c.NotebookApp.open_browser = False
...

By default, JupyterLab attempts to automatically initiate a browser session when it starts running. Since we do not have a browser on the remote server, it is necessary to turn this off to avoid errors.

Next, navigate down to the c.NotebookApp.password variable and change to the following:

...
## Hashed password to use for web authentication.
#
#  To generate, type in a python/IPython shell:
#
#    from notebook.auth import passwd; passwd()
#
#  The string should be of the form type:salt:hashed-password.
c.NotebookApp.password = 'your_hashed_password'
...

JupyterLab will use this hashed password configuration to check the password you enter for access in your browser.

Finally, navigate further through the file and update the entry of the c.NotebookApp.port:

...
## The port the notebook server will listen on.
c.NotebookApp.port = 9000
...

c.NotebookApp.port sets a fixed port for accessing your JupyterLab runtime. This way, you can allow access for only one port from the ufw firewall.

Once you’re done, save and exit the file.

Finally, allow traffic on the 9000 port:

sudo ufw allow 9000

You’ll receive the following output:

Output
Rule added
Rule added (v6)

Now that you’ve set all your configuration, you’ll run JupyterLab.

Step 4 — Running JupyterLab

In this step, you’ll perform a test run of the JupyterLab instance.

First, change your current working directory to the user’s home directory:

cd ~

Now, modify the access permissions of the certificate files to allow JupyterLab to access them. Change the permissions of the /etc/letsencrypt folder to the following:

sudo chmod 750 -R /etc/letsencrypt
sudo chown sammy:sammy -R /etc/letsencrypt

Then, invoke your JupyterLab instance to start using the following command:

jupyter lab

This command accepts several configuration parameters. However, since we have already made these changes in the configuration file, we do not need to provide them here explicitly. You can provide them as arguments to this command to override the values in the configuration file.

You can now navigate to https://your_domain:9000 to check you receive JupyterLab’s login screen.

If you log in with the password you set up for JupyterLab in Step 2, you’ll be presented with the JupyterLab interface.

Finally, press CTRL+C twice to stop the JupyterLab server.

In the next step, you’ll set up a system service so that the JupyterLab server can be run in the background continuously.

Step 6 — Setting Up a systemd Service

In this step, you will create a systemd service that allows JupyterLab to keep running even when the terminal window is exited. You can read more about systemd services and units in this guide on systemd essentials.

First, you’ll have to create a .service file, using the following command:

sudo nano /etc/systemd/system/jupyterlab.service

Add the following content to the /etc/systemd/system/jupyterlab.service file:

[Unit]
Description=Jupyter Lab Server

[Service]
User=sammy
Group=sammy
Type=simple
WorkingDirectory=/home/sammy/
ExecStart=/home/sammy/anaconda3/bin/jupyter-lab --config=/home/sammy/.jupyter/jupyter_notebook_config.py
StandardOutput=null
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Save and exit the editor once you’re done.

The service file automatically registers itself in the system as a daemon. However, it is not running by default.

Use the systemctl command to start the service:

sudo systemctl start jupyterlab

This starts the JupyterLab server in the background. You can check if the server has started by using the following command:

sudo systemctl status jupyterlab

You’ll receive the following output:

Output
● jupyterlab.service - Jupyter Lab Server
   Loaded: loaded (/etc/systemd/system/jupyterlab.service; disabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-04-26 20:58:29 UTC; 5s ago
 Main PID: 5654 (jupyter-lab)
    Tasks: 1 (limit: 1152)
   CGroup: /system.slice/jupyterlab.service
           └─5654 /home/sammy/anaconda3/bin/python3.7 /home/sammy/anaconda3/bin/jupyter-lab --config=/home/

Press Q to exit the service status output.

You can now head to https://your_domain:9000 in any browser of your choice, provide the password you set up in Step 2, and access the JupyterLab environment running on your server.

Step 7 — Configuring Renewal of Your Let’s Encrypt Certificate

In this final step, you will configure your SSL certificates provided by Let’s Encrypt to automatically renew when they expire every 90 days and then restart the server to load the new certificates.

While Certbot takes care to renew the certificates for your installation, it does not automatically restart the server. To configure the server to restart with new certificates, you will have to provide a renew_hook to the Certbot configuration for your server.

You’ll need to edit the /etc/letsencrypt/renewal/your_domain.conf file and add a renew_hook to the end of the configuration file.

First, use the following command to open the /etc/letsencrypt/renewal/your_domain.conf file in an editor:

sudo nano /etc/letsencrypt/renewal/your_domain.conf

Then, at the bottom of this file, add the following line:

...
renew_hook = systemctl reload jupyterlab

Save and exit the file.

Finally, run a dry-run of the renewal process to verify that your configuration file is valid:

sudo certbot renew --dry-run

If the command runs without any error, your Certbot renewal has been set up successfully and will automatically renew and restart your server when the certificate is near the date of expiry.

Conclusion

In this article, you set up a JupyterLab environment on your server and made it accessible remotely. Now you can access your machine learning or data science projects from any browser and rest assured that all exchanges are happening with SSL encryption in place. Along with that, your environment has all the benefits of cloud-based servers.