GPU resources

List nodes with GPU resources

List the resources on nodes with access to Graphics Processing Units (GPUs)

kubectl get nodes -o=go-template='{{range .items}}{{$c := .status.capacity}}{{$gpus := index $c "nvidia.com/gpu"}}{{if $gpus}}{{if gt $gpus "0"}}{"{{.metadata.name}}":{"cpu":{{$c.cpu}},"gpu":{{$gpus}},"mem":"{{$c.memory}}","hdd":"{{index $c "ephemeral-storage"}}","accelerator":"{{.metadata.labels.accelerator}}"{{"}}\n"}}{{end}}{{end}}{{end}}'

{"p02r01srv02":{"cpu":64,"gpu":8,"mem":"263751564Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r01srv03":{"cpu":64,"gpu":8,"mem":"263751548Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r01srv04":{"cpu":64,"gpu":8,"mem":"263751568Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r01srv05":{"cpu":64,"gpu":8,"mem":"263751568Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r01srv06":{"cpu":64,"gpu":8,"mem":"263751572Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r01srv07":{"cpu":64,"gpu":8,"mem":"263751556Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv01":{"cpu":64,"gpu":8,"mem":"263751580Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv02":{"cpu":64,"gpu":8,"mem":"263751568Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv03":{"cpu":64,"gpu":8,"mem":"263751576Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv04":{"cpu":64,"gpu":8,"mem":"263751552Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv05":{"cpu":64,"gpu":8,"mem":"263751572Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r02srv06":{"cpu":64,"gpu":8,"mem":"263764620Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r02srv07":{"cpu":64,"gpu":8,"mem":"263764628Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r08srv01":{"cpu":384,"gpu":10,"mem":"1584880532Ki","hdd":"921225988Ki","accelerator":"nvidia-h100"}}
{"p02r08srv02":{"cpu":384,"gpu":10,"mem":"1584880612Ki","hdd":"921191180Ki","accelerator":"nvidia-h100"}}
{"p02r08srv03":{"cpu":384,"gpu":10,"mem":"1584880988Ki","hdd":"921217796Ki","accelerator":"nvidia-h100"}}
{"p02r09srv01":{"cpu":64,"gpu":8,"mem":"263751584Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r09srv02":{"cpu":64,"gpu":8,"mem":"263751584Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r09srv03":{"cpu":64,"gpu":8,"mem":"263764628Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r09srv04":{"cpu":64,"gpu":8,"mem":"263764604Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r09srv05":{"cpu":64,"gpu":8,"mem":"263764624Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r09srv06":{"cpu":64,"gpu":8,"mem":"263764604Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-1080ti"}}
{"p02r09srv07":{"cpu":64,"gpu":7,"mem":"263759716Ki","hdd":"1844812632Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r10srv02":{"cpu":64,"gpu":8,"mem":"263751568Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r10srv03":{"cpu":64,"gpu":8,"mem":"263751564Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r10srv04":{"cpu":64,"gpu":8,"mem":"263751576Ki","hdd":"960302804Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r10srv05":{"cpu":64,"gpu":8,"mem":"263751552Ki","hdd":"960167960Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r10srv06":{"cpu":64,"gpu":8,"mem":"263751572Ki","hdd":"960167960Ki","accelerator":"nvidia-gtx-2080ti"}}
{"p02r99srv01-a16":{"cpu":48,"gpu":8,"mem":"263913512Ki","hdd":"203056560Ki","accelerator":"nvidia-a16"}}

Available GPUs

You have to use nodeSelector/accelerator to specify the type of GPU(s) to assign to your pod.

Type	Description	Memory
nvidia-gtx-1080ti	Nvidia GTX 1080ti	11GB
nvidia-gtx-2080ti	Nvidia GTX 2080ti	11GB
nvidia-a16	Nvidia A16	16GB
nvidia-h100	Nvidia H100 NVL	94GB

Request GPU resources

Following YAML file demonstrates how to request one Nvidia GTX 2080ti GPU:

gpu-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - name: cuda
    image: nvidia/cuda:12.2.0-base-ubuntu22.04
    command:
      - sleep
      - infinity
    resources:
      requests:
        cpu: "500m"
        memory: "2Gi"
        nvidia.com/gpu: "1"
      limits:
        memory: "2Gi"
        nvidia.com/gpu: "1"
  nodeSelector:
    accelerator: nvidia-gtx-2080ti

Create the pod and the GPU will be directly accessible

kubectl create -n your_ns -f gpu-pod.yaml
kubectl exec -n your_ns testpod -- sh -c 'nvidia-smi -L'

GPU 0: GeForce RTX 2080 Ti (UUID: GPU-cd469f06-a7da-e73c-70fe-161dc35b587b)