Configure ArcGIS Notebook Server to use GPUs

The use of graphical processing units (GPUs) rapidly improves the processing time for machine learning models. ArcGIS Notebook Server can take advantage of NVIDIA GPUs on its host machine once some additional steps are performed.

Note:

Starting at 10.8, the built-in ArcGIS Notebook Server runtimes include the Conda CUDA Toolkit to enable GPU support. Previously, this workflow required building a custom runtime to include CUDA.

The following workflow has two primary goals. The first is to install NVIDIA drivers and runtime, which will allow your site's Docker component to build GPU-ready containers. The second is to create a copy of the notebook runtime that's configured to use the NVIDIA runtime. All notebooks opened using this runtime will launch in GPU-ready containers. Aside from that, the new runtime will keep all the Python libraries of the notebook runtime.

Once ArcGIS Notebook Server has been installed and configured, follow these steps. If your ArcGIS Notebook Server site has multiple machines, follow steps 1 through 3 on all machines.

1. Install the appropriate NVIDIA drivers on each machine in your site. See the NVIDIA website for complete information.
2. Install the nvidia-docker 2.0 runtime on the machine so that notebook containers can take advantage of GPUs. Refer to the NVIDIA-Docker repository on GitHub for the downloads and documentation pertaining to your specific OS.
3. Run the following command on each machine to ensure that your NVIDIA elements are properly installed:

   docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi

4. Sign in to your ArcGIS Enterprise portal as an administrator and open ArcGIS Notebook Server Manager.
5. Open the Settings page, and click Runtimes.
6. Click the Edit button for ArcGIS Notebook Python 3 Advanced or ArcGIS Python 3 Standard, depending on the runtime you will use as the base of the GPU-enabled runtime. Copy the value given for its Image ID. Click Cancel to exit the editor.
7. From the Runtimes page, click Register Runtime.
8. On the Register Runtime page, provide an appropriate name (such as GPU Runtime) and give the version as 10.9.1. For the Image ID value, add the value you copied in step 5.
9. Set the Docker Runtime value to nvidia. Click Register Runtime to confirm.
10. Verify you have successfully configured ArcGIS Notebook Server to use NVIDIA GPUs. As a portal member with the Create and edit notebooks privilege or the Advanced Notebooks privilege if you chose the advanced runtime in Step 6, create a new blank notebook. When you choose the runtime of the notebook, select your new GPU-ready runtime. Copy the following into a notebook cell and run the cell.

    import torch
    torch.cuda.is_available()

    The output returns as True, because the torch.cuda package requires GPUs to run.

11. Run the following command in a new cell to view your machine's GPU configuration:

    !nvidia-smi

To remove the capacity for your site to use GPUs, go to the Runtimes section on the Settings page in ArcGIS Notebook Server Manager.