Space Allocation of a Data Disk

Allocating Default Data Disk Space

• Space Allocation for Container Engines
  • Specified disk space: CCE divides the data disk space for two parts by default. One part is used to store the Docker/containerd working directories, container image data, and image metadata. The other is reserved for kubelet and emptyDir volumes. The available container engine space affects image pulls and container startup and running.
    • Container engine and container image space (90% by default): stores the container runtime working directories, container image data, and image metadata.
    • kubelet and emptyDir space (10% by default): stores pod configuration files, secrets, and mounted storage such as emptyDir volumes.
    Note

    If the sum of the container engine and container image space and the kubelet and emptyDir space is less than 100%, the remaining space will be allocated for user data. You can mount the storage volume to a service path. Do not leave the path empty or set it to a key OS path such as the root directory.

  • Shared disk space: In clusters of v1.21.10-r0, v1.23.8-r0, v1.25.3-r0, or later versions, CCE allows a container engine (Docker/containerd) and kubelet components to share data disk space.
• Space Allocation for Pods: indicates the basesize of a pod. You can set an upper limit for the disk space occupied by each workload pod (including the space occupied by container images). This setting prevents the pods from taking all the disk space available, which may cause service exceptions. It is recommended that the value is less than or equal to 80% of the container engine space. This parameter is related to the node OS and container storage Rootfs and is not supported in some scenarios. For details, see Mapping Between OS and Container Storage Rootfs.
• Write Mode
  • Linear: A linear logical volume integrates one or more physical volumes. Data is written to the next physical volume when the previous one is used up.
  • Striped: available only if there are at least two data disks. A striped logical volume stripes data into blocks of the same size and stores them in multiple physical volumes in sequence. This allows data to be concurrently read and written. A storage pool consisting of striped volumes cannot be scaled-out.

Space Allocation for Container Engines

• Rootfs (Device Mapper)

  By default, the container engine and image space, occupying 90% of the data disk, can be divided into the following two parts:

  • The /var/lib/docker directory is used as the Docker working directory and occupies 20% of the container engine and container image space by default. (Space of the /var/lib/docker directory = Data disk space × 90% × 20%)
  • The thin pool is used to store container image data, image metadata, and container data, and occupies 80% of the container engine and container image space by default. (Thin pool space = Data disk space × 90% × 80%)

    The thin pool is dynamically mounted. You can view it by running the lsblk command on a node, but not the df -h command.

  Figure 1 Space allocation for container engines of Device Mapper

  
  

• Rootfs (OverlayFS)

  No separate thin pool. The entire container engine and container image space (90% of the data disk by default) are in the /var/lib/docker directory.

  Figure 2 Space allocation for container engines of OverlayFS

  
  

Space Allocation for Pods

• Device Mapper supports custom pod basesize. The default value is 10 GiB.
• In OverlayFS mode, the pod container space is not limited by default.

Mapping Between OS and Container Storage Rootfs

Garbage Collection Policies for Container Images

Recommended Configuration for the Container Engine Space

• The container engine space should be greater than the total disk space used by containers. Formula: Container engine space > Number of containers x basesize
• You are advised to create and delete files of containerized services in local storage volumes (such as emptyDir and hostPath volumes) or cloud storage directories mounted to the containers. In this way, the thin pool space is not occupied. emptyDir volumes occupy the kubelet space. Therefore, properly plan the size of the kubelet space.
• You can deploy services on nodes that use the OverlayFS (for details, see Mapping Between OS and Container Storage Rootfs) so that the disk space occupied by files created or deleted in containers can be released immediately.

Data Disk Shared Between a Container Engine and kubelet Components

• dockersys: /mnt/paas/runtime
• Kubernetes: /mnt/paas/kubernetes/kubelet

Figure 3 Allocating the storage space of a shared data disk