Extend a notebook runtime

ArcGIS Notebook Server includes two notebook runtimes (collections of Python modules). These runtimes are available to notebook users through container images, which are applied to each container that runs in your server site.

Both the Standard and Advanced notebook runtimes include ArcGIS API for Python and various Python modules that can be used for data science, spatial analysis, content management, and administration. The Advanced runtime also includes ArcPy.

See the list of available Python libraries

In some cases, your notebook authors may need modules or specific versions of modules that are not included in these two runtimes. You can access additional Python modules in ArcGIS Notebook Server in the following ways:

• Install a Python module during a notebook session. This makes the module available for use in that notebook.
• Extend an Esri container image to include additional Python libraries in a custom runtime. The container image you create will make the extended runtime available to all notebook authors in your site.

Install modules during a session

Notebook authors can make Python modules, or versions of modules, that are not included in either runtime available for use during a notebook session using the conda or pip package management systems. It is recommended that you use conda, because ArcGIS Notebook Server uses it to manage modules; however, some Python modules are only available in pip.

This approach installs the desired module for the duration of the notebook session. Once the notebook is closed or the container belonging to the user is terminated, the module is no longer available. To install modules using this method, users must have internet connectivity to the conda or pip channel.

Note:

Both conda and pip are included in the two notebook runtimes. You do not need to install them.

To install the scrapy package during a session, for example, run a notebook cell with one of the following commands, depending on the package management system you're using:

Using the conda package management system

!conda install --yes scrapy

Using the pip package management system

!pip install scrapy

To use a different version of a Python module in the notebook runtime you're working in, specify the version number in the command.

You must restart the kernel after running a conda or pip command for your new package to be available.

For example, a certain package is available in both notebook runtimes at version 1.5, but you want to use a tool introduced in 1.6. To make version 1.6 of the package available during a notebook session, run a notebook cell with one of the following commands, depending on the package management system you're using:

Using the conda package management system

!conda install --yes <package name>==1.6

Using the pip package management system

!pip install <package name>==1.6

Build a custom container image

To make persistent changes to a notebook runtime, you can build a custom container image in a site. The following workflow, which requires administrative privileges, creates a custom image and configures ArcGIS Notebook Server to use it.

Use either of the two notebook runtimes (Standard or Advanced) as a starting point. Once you have built your custom runtime, you can register it as a new runtime in your site.

1. Install and configure ArcGIS Notebook Server.
2. Sign in to the ArcGIS Notebook Server Administrator Directory as an administrator.

   The URL is formatted as https://notebookserver.domain.com:11443/arcgis/admin/.

3. Click notebooks > runtimes and select the notebook runtime you want to extend for your custom image.

   On the runtime's resource page, note the imageId value for this runtime. You will include this value in the next step.

4. Create a Dockerfile that adds your desired Python modules to the custom image.

   A Dockerfile is a simple text file of instructions to build a Docker container image. Follow the Dockerfile documentation for information and best practices.

   In this example, you'll add the data visualization Python package graphviz to the Advanced notebook runtime. The following code block in your Dockerfile instructs Docker to access the Advanced runtime using its imageId value and to install graphviz in the runtime. The conda clean command is a best practice to reduce file size.

   # Specify the existing notebook runtime imageId as FROM
   FROM aa7a1a346e5b
   
   # Use RUN to issue a command to install, then clean
   RUN conda install graphviz \
     && conda clean -y -a

   In this example, you will create a custom runtime that includes Oracle client libraries.

   ## Using a Dockerfile to build a custom notebook runtime that includes Oracle client
   FROM <replace with imageId of standard or advanced container image>
   
   ## Switch to root user for installing OS level packages. 
   USER root
   
   ## Combine OS level statements for Docker image efficiency
   RUN apt-get update && \
           apt-get install -y alien
   ## Update the url below to the desired version of Oracle client.
   RUN wget https://yum.oracle.com/repo/OracleLinux/OL8/oracle/instantclient/x86_64/getPackage/oracle-instantclient19.10-basic-19.10.0.0.0-1.x86_64.rpm
   RUN alien -i oracle-instantclient19.10-basic-19.10.0.0.0-1.x86_64.rpm
   
   ## Installing libaio1 is required only for Notebook runtimes version 5 (10.9) or earlier.  
   RUN apt-get install -y libaio1
   
   ## Switch back to arcgis user at end of the Dockerfile
   USER arcgis

5. Save your Dockerfile on the machine.
6. Build your new custom container image using a Docker command.

   This image will contain a new runtime as specified in your Dockerfile. Open a terminal and run a command with the following syntax:

   docker build -t <name of my new runtime>:v1.0 -f <path to my Dockerfile> .

   Note:

   Be sure to include the period at the end of the command.

   When the command executes, Docker builds the new container image. The last line in the command output will be Successfully built <imageId> with the abridged ID value of the new image. You must provide your ArcGIS Notebook Server site with its full imageId value.

7. Run the following command in your terminal to get the image ID value:

   docker inspect <imageId>

   Now that your custom image is built and you have its full imageId value, return to the ArcGIS Notebook Server Administrator Directory to register the new image.

8. In the Administrator Directory, browse to notebooks > runtimes. Click Register Runtime.
9. Specify the appropriate values to complete the registration.

   A guide to these parameters is given in the Register runtime reference topic. When you're ready, register the runtime with the site.

10. Open a new notebook to test this update. Verify that the modules or versions of modules you added to your custom image are available in the notebook.

To return the runtimes in your ArcGIS Notebook Server site to the original settings, use the Restore Factory Runtimes operation in the Administrator Directory. This is available at https://notebookserver.domain.com:11443/arcgis/admin/notebooks/runtimes/restore, and will only restore the factory runtimes for your site's current version.