GPU Support

Using GPUs On The Grid

We have recently added some grid nodes with Nvidia GA100 GPUs. Since worker nodes typically have a limited selection of installed software, jobs which use GPUs will need to prepare an environment for their use.

Anaconda Example

The following example is based on the Anaconda python distribution and some familiarity with this is probably desirable. Through Anaconda we can obtain "cudatoolkit" which provides support for the GPU and "numba" which is python library that you can use to make use of the GPU.

[
JobName = "gpu_test";
Executable = "gpu_test.sh";
Arguments = "";
StdOutput = "StdOut";
StdError = "StdErr";
InputSandbox = {"gpu_test.sh","gpu_test.py","LFN:/gridpp/user/d/dan.whitehouse/Anaconda3-2022.05-Linux-x86_64.sh"};
OutputSandbox = {"StdOut","StdErr"};
Site = "LCG.UKI-LT2-IC-HEP.uk";
Tags = {"GPU"}
]

Our bash script is set as the executable in the JDL above and contains:

#!/bin/sh
./Anaconda3-2022.05-Linux-x86_64.sh -p ${PWD}/gputest -b
source ${PWD}/gputest/etc/profile.d/conda.sh
conda info -e
conda activate base
conda install cudatoolkit numba
./gpu_test.py

Here we install Anaconda using the script that we have downloaded and then referenced in our InputSandbox. This is simply the default installation script you can download from the Anaconda website. In the JDL this has been uploaded to my personal share in the gridpp section of our SE - we do this rather than uploading with the job since the Anaconda installer is quite large. Once installed we can list our python environments and then activate the "base" environment. We then install the cudatoolkit so we can make use of the GPU, and the "numba" package. With those dependencies installed in our environment, we can then execute our python script which simply contains:

#!/usr/bin/env python
from numba import cuda
print(cuda.gpus)

If we submit the job and look at the last excerpt of our output:

<Managed Device 0>

We can see that we can indeed access our GPU using python.