Volumes

Persistent storage volumes are provided through Ceph Block Devices and can be attached to containers as Kubernetes Volumes. They are not automatically deleted along with the container.

There are two storage classes to choose from:

• rook-ceph-rbd for standard fast storage accessed by a single container
• rook-ceph-fs for containers that need shared storage

The volumes can have the following accessModes:

• ReadWriteOnce means only a single container can access the volume.
• ReadWriteMany for network file system access between multiple containers.
• ReadOnlyMany for constant data which is defined when the volume is created.

I.e. use rook-ceph-fs for ReadWriteMany and it is recommended to use rook-ceph-rbd for ReadWriteOnce.

Standard Fast Storage Example

Following is an example of storage class rook-ceph-rbd and access mode ReadWriteOnce. Use this for fast and exclusive access to a persistent storage volume.

storage.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: testpod-storage
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 1Gi
  storageClassName: rook-ceph-rbd

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - name: bb
    image: busybox
    command:
      - sleep
      - infinity
    resources:
      requests:
        memory: "128Mi"
        cpu: "500m"
      limits:
        memory: "128Mi"
    volumeMounts:
    - name: exclusive-volume
      mountPath: /data
  volumes:
  - name: exclusive-volume
    persistentVolumeClaim:
      claimName: testpod-storage

Create the volume and the pod

kubectl create -f storage.yaml
kubectl create -f pod.yaml

Try writing to the volume, re-creating the pod and then reading

kubectl exec testpod -- sh -c "echo 'Hello, world\!' >> /data/hello"
kubectl delete testpod
kubectl create -f pod.yaml
kubectl exec testpod -- sh -c "cat /data/hello"

Hello, world!

Shared Storage Example

Following is an example of storage class rook-ceph-fs and access mode ReadWriteMany. This will allow multiple containers to mount the storage volume and modify its contents.

shared-storage.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: testpods-storage
spec:
  accessModes:
  - ReadWriteMany
  volumeMode: Filesystem
  resources:
    requests:
      storage: 1Gi
  storageClassName: rook-ceph-fs

shared-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: podname
spec:
  containers:
  - name: bb
    image: busybox
    command:
      - sleep
      - infinity
    resources:
      requests:
        memory: "128Mi"
        cpu: "500m"
      limits:
        memory: "128Mi"
    volumeMounts:
    - name: shared-volume
      mountPath: /shared
  volumes:
  - name: shared-volume
    persistentVolumeClaim:
      claimName: testpods-storage

Create the volume, and two pods sharing it

kubectl create -f shared-storage.yaml
sed "s/podname/sharedpod0/" shared-pod.yaml | kubectl create -f -
sed "s/podname/sharedpod1/" shared-pod.yaml | kubectl create -f -

Try writing to the volume from one pod and reading from the other

kubectl exec sharedpod0 -- sh -c "echo 'Hello, world\!' >> /shared/hello"
kubectl exec sharedpod1 -- sh -c "cat /shared/hello"

Hello, world!