GPU servers on ECC
The following applies to both virtual and bare-metal GPU servers running Ubuntu 22.04.
Install Nvidia drivers on Ubuntu
To use the Nvidia GPU, you need to install its drivers.
After rebooting, you can check that the GPU is working by running
Disable default Ubuntu services
When using Nvidia GPUs for hardware acceleration in CUDA or VNC you might want to disable the automatic updates and power-saving features.
Disable automatic updates
When running nvidia-smi, you might see a message like this:
This typically happens because the Nvidia driver was updated automatically, while the Nvidia Management Library (NVML) was not (or vice versa). It can be fixed by rebooting the server.
You can disable automatic updates with
sudo systemctl stop apt-daily.service
sudo systemctl stop apt-daily.timer
sudo systemctl stop apt-daily-upgrade.service
sudo systemctl stop apt-daily-upgrade.timer
sudo systemctl disable apt-daily.service
sudo systemctl disable apt-daily.timer
sudo systemctl disable apt-daily-upgrade.service
sudo systemctl disable apt-daily-upgrade.timer
Just remember to run sudo apt update && sudo apt upgrade manually from time to time.
Disable power saving
The hibernation feature can cause issues accessing the GPU, so you may want to disable it, along with the other power-saving features.
sudo systemctl stop nvidia-suspend.service
sudo systemctl stop nvidia-hibernate.service
sudo systemctl stop nvidia-resume.service
sudo systemctl disable nvidia-suspend.service
sudo systemctl disable nvidia-hibernate.service
sudo systemctl disable nvidia-resume.service
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target
Remote desktop with hardware acceleration
If you need to run a graphical application on the server, you can use Virtual Network Computing (VNC) to access the server's desktop with hardware acceleration.
VNC
Virtual Network Computing (VNC) is a type of remote desktop for running graphical applications on a remote server. GPU servers can be configured to enable server-side hardware-accelerated 3D rendering for OpenGL graphics software, such as Blender, MATLAB 3D simulations or fluid dynamics programs. The remote GPU renders the 3D image and streams it to your laptop.
VirtualGL
VirtualGL redirects an application's OpenGL/GLX commands to a separate X server (that has access to a 3D graphics card), captures the rendered images, and then streams them to the X server that handles the application. Framerate performance depends largely on network speed.
Installation
Login to the server by SSH, and port forward remote screen 1 (port 5901) to localhost screen 2 (port 5902)
Set your timezone
Install the latest Nvidia drivers, VirtualGL and TurboVNC, then add them to the executable path.
sudo apt update && sudo apt upgrade
sudo apt install nvidia-headless-525 nvidia-utils-525 \
libglu1-mesa libegl1-mesa libxv1 libxtst6 libxt6 x11-xkb-utils pkg-config
wget -O virtualgl.deb "https://downloads.sourceforge.net/project/virtualgl/3.1/virtualgl_3.1_amd64.deb"
wget -O turbovnc.deb "https://downloads.sourceforge.net/project/turbovnc/3.0.3/turbovnc_3.0.3_amd64.deb"
sudo dpkg -i virtualgl.deb turbovnc.deb
sudo bash -c 'echo "PATH=/opt/TurboVNC/bin:/opt/VirtualGL/bin:${PATH}" > /etc/profile.d/99-vgl-path.sh'
Reboot the server to load the drivers since nvidia-xconfig needs to access the GPU. If you already have a working /etc/X11/xorg.conf, you can skip this.
After reboot, check that nvidia-smi finds a GPU, then configure VirtualGL.
sudo nvidia-xconfig
sudo modprobe -rf nvidia_drm nvidia_modeset nvidia_uvm nvidia
sudo vglserver_config +glx
sudo usermod -aG vglusers ubuntu
Install a desktop and remove any screensavers
sudo apt install xfce4 xfce4-goodies lightdm lightdm-gtk-greeter xorg mesa-utils nvidia-settings
sudo apt -y remove light-locker xscreensaver xfce4-screensaver
sudo systemctl set-default graphical.target
Reboot the server to load the drivers, reload user groups, and start various system services.
Start the VNC server
Check that the server is running
If the server does not start, check the logs in /home/ubuntu/.vnc/ and /var/log/Xorg.0.log for errors. You may need to manually load the nvidia module with sudo modprobe nvidia.
You can now connect to the server using a VNC client, such as TigerVNC. Since you are forwarding port 5901 to port 5902 on localhost, you need to connect to VNC Server: localhost:2.
From the VNC client, start a terminal and check that direct rendering is working:
You can also check that the GPU is working by running the Basemark GPU benchmark.
