Setting Up HIP with CUDA

WIP 1: I'm trying to make a Docker image of ROCm stack for NVIDIA GPUs, but it's somewhat problematic. I'll update this post as I make progress. WIP 2: Among the scripts below, there would be some that are not necessary. I'll clean them up later.

HIP / ROCm / CUDA

HIP is like a thin layer on top of ROCm or CUDA. However, HIP is often regarded a part of ROCm stack, and that's where problem rises: getting the hip packages often leads to installing other ROCm components that are not compatible with NVIDIA stack.

System Requirements

I'm using Ubuntu 22.04.5 LTS 64bit, and it might or might not work on Ubuntu 24.04. I've experienced a kernel update breaking the whole system, so as of now, I recommend using Ubuntu 22.04.

AMD GPU: ROCm + CUDA Toolkit

It works without any problem. Just install ROCm and CUDA Toolkit on the same system. However, when installing CUDA Toolkit, make sure you omit the driver installation and set the version to the one supported by hipify. In case of ROCm 6.2.0, install CUDA 12.3.

Below is the list of commands needed, taken from ROCm installation guide and CUDA installation guide.

Below commands are taken from the official installation guides, so there's nothing much to explain.

Installing ROCm

sudo apt update
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo usermod -a -G render,video $LOGNAME # Add the current user to the render and video groups
wget https://repo.radeon.com/amdgpu-install/6.2/ubuntu/jammy/amdgpu-install_6.2.60200-1_all.deb
sudo apt install ./amdgpu-install_6.2.60200-1_all.deb
sudo apt update
sudo apt install rocm
# Actually, the official documentation says to install rocm-dkms, but it causes some troubles on my system.

Installing CUDA Toolkit

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.3.0/local_installers/cuda-repo-ubuntu2204-12-3-local_12.3.0-545.23.06-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-3-local_12.3.0-545.23.06-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-3-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt update
sudo apt -y install cuda-toolkit-12-3 # Don't install 'cuda'!

However, if you want to build a HIP program for an NVIDIA GPU target, CMake should be updated to the latest version(3.30.2 in my case), some environment variables should be set, and hipblas package from the package manager doesn't work. Just calling hipcc commands for a single file or make will work, but CMake will fail.

Installing CMake

sudo apt update
sudo apt install ca-certificates gpg wget
test -f /usr/share/doc/kitware-archive-keyring/copyright ||
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | sudo tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null
echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ jammy main' | sudo tee /etc/apt/sources.list.d/kitware.list >/dev/null
sudo apt update
sudo apt install cmake

Regarding hipBLAS, get it directly from the repository and install it manually.

# After you've donwloaded it, uncompressed it and cd into the directory
./install.sh -i

Regarding the environment variables, I've set the following in my .bashrc file.

export PATH=$PATH:/opt/rocm/bin
export PATH=$PATH:/usr/local/cuda/bin

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib

and if you want to use hipcc to compile HIP programs for NVIDIA GPUs, you should set the following environment variable.

export HIP_PLATFORM=nvidia
export HIP_COMPILER=nvcc
export HIP_RUNTIME=cuda

NVIDIA GPU: HIP + CUDA Toolkit

First, setup CUDA as usual, but like above, use the right version. Trying to installing hipcc-nvidia from ROCm 6.1 repository won't work. There's a bug in the version, and you should use ROCm 6.2. Add the ROCm repository and install hipcc-nvidia.

Installing CUDA Toolkit

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.3.0/local_installers/cuda-repo-ubuntu2204-12-3-local_12.3.0-545.23.06-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-3-local_12.3.0-545.23.06-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-3-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt update
sudo apt -y install cuda-toolkit-12-3

Installing HIP

sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
    gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

# Register ROCm packages
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.2 jammy main" \
    | sudo tee --append /etc/apt/sources.list.d/rocm.list
echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' \
    | sudo tee /etc/apt/preferences.d/rocm-pin-600
sudo apt update
# Don't install the kernel driver or the whole ROCm stack!

sudo apt install hip-runtime-nvidia hip-dev

Other than that, install CMake, set the environment variables, and install hipBLAS manually as above.

CMake Flags for Hipcc

When using hipcc for NVIDIA GPUs, you should add the following flags to CMake. I suppose it doesn't have to be this redundant, but after all the errors, I chose to let it be.

-DCMAKE_HIP_COMPILER=nvcc
-DCMAKE_GPU_RUNTIME=CUDA
-DCMAKE_HIP_ARCHITECTURES=70 # or whatever your GPU architecture is. For Ada Lovelace (RTX 4050 ~ RTX 4090), it's 89.

And as it's passed to nvcc, you should not set flags like --offload-arch like you do with AMD GPUs.