Yesterday I upgraded to the latest version of bumblebee only to learn EXLA took on the latest XLA which dropped support for CUDA 11.2.
Because I was running Pop!OS I assumed it would be something I could `apt install` and call it good. Unfortunately it was more work than my first go round so I wanted to detail it here for anyone who might follow. This is loosely based off the older CUDA 11 install instructions I found but updated for ubuntu 22.04.
Before you get started downloading anything, setup a developer account with nvidia so you can download cuDNN in a few minutes.
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
$ sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
$ sudo add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
$ sudo apt update
$ sudo apt install cuda-toolkit-12-2
Next download cuDNN from nvidia and install it.
$ sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.4.25_1.0-1_amd64.deb
$ sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.4.25/cudnn-local-72322D7F-keyring.gpg /usr/share/keyrings/
$ sudo apt update
$ sudo apt install libcudnn8=8.9.4.25-1+cuda12.2
Finally, set a few env variables and the correct XLA_TARGET for EXLA
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
export CUDA_HOME=/usr/local/cuda
export PATH="/usr/local/cuda/bin:$PATH"
export XLA_TARGET=cuda120
export XLA_FLAGS=--xla_gpu_cuda_data_dir=/usr/local/cuda
Open a new terminal session and verify CUDA 12 is setup correctly by running this command
nvcc -V
