Configure ArcGIS Notebook Server to use GPUs

The use of graphical processing units (GPUs) rapidly improves the processing time for machine learning models. ArcGIS Notebook Server can take advantage of NVIDIA GPUs on its host machine once additional steps are performed. The following workflow has two primary goals. The first is to install NVIDIA drivers and runtime, which will allow your site's Docker component to build GPU-ready containers. The second is to create a copy of the notebook runtime that's configured to use the NVIDIA runtime. All ArcGIS Notebooks notebooks opened using this runtime will launch in GPU-ready containers. Aside from that, the new runtime will keep all the Python libraries of the notebook runtime.

Once ArcGIS Notebook Server has been installed and configured, follow these steps. If your ArcGIS Notebook Server site has multiple machines, follow steps 1 through 3 on all machines.

1. Install the appropriate NVIDIA drivers on each machine in your site. See the NVIDIA CUDA Installation Guide for Linux for complete information.
   1. Perform Post-installation actions to set up the environment after installing the CUDA Toolkit.
2. Install the nvidia-container-toolkit on the machine so that notebook containers can take advantage of GPUs. Refer to the NVIDIA Container Toollkit Installation guide for the downloads and documentation pertaining to your specific OS.
3. Run the following command on each machine to ensure that your NVIDIA elements are properly installed:

   sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

4. Sign in to your ArcGIS Enterprise portal as an administrator and open ArcGIS Notebook Server Manager.
5. Open the Settings page, and click Runtimes.
6. Click the Edit button for the runtime you will use as the base of the GPU-enabled runtime, either ArcGIS Notebook Python 3 Advanced or ArcGIS Notebook Python 3 Standard. Copy the Image ID value. Click Cancel to exit the editor.
7. From the Runtimes page, click Register runtime.
8. On the Register runtime page, provide an appropriate name (such as GPU Runtime). For the Image ID value, add the value you copied in step 6.
9. Set the Docker runtime value to nvidia. Click Register runtime to confirm.
10. Verify you have successfully configured ArcGIS Notebook Server to use NVIDIA GPUs. As a portal member with the Create and edit notebooks privilege or the Advanced notebooks privilege, if you chose the advanced runtime in step 6, create a blank notebook. When you choose the runtime of the notebook, select your new GPU-ready runtime. Copy the following into a notebook cell and run the cell.

    import torch
    torch.cuda.is_available()

    The output returns as True, because the torch.cuda package requires GPUs to run.

11. Run the following command in a new cell to view your machine's GPU configuration:

    !nvidia-smi

If you want to remove the capacity for your site to use GPUs, go to the Runtimes section on the Settings tab in ArcGIS Notebook Server Manager and delete the runtime you created in this workflow.