Using a Custom Access Key (AK/SK) to Mount an OBS Volume

• The CCE Container Storage (Everest) version must be 1.2.8 or later.
• The cluster version must be 1.15.11 or later.

• When an OBS volume is mounted using custom access keys (AK/SK), the access key cannot be deleted or disabled. Otherwise, the service container cannot access the mounted OBS volume.
• Custom access keys cannot be configured for secure containers.

When creating an OBS volume on the console of the old edition, you need to upload the AK/SK, which are global access keys used for mounting OBS volumes. As a result, all IAM users in your account will use the same keys to mount OBS buckets as volumes, and they will have identical permissions on the buckets. However, this setting does not allow you to set different permissions for individual IAM users.

If you have uploaded the AK/SK (specifically, if paas.longaksk exists in the kube-system namespace of the cluster), you should disable the global access secret to prevent IAM users from performing unauthorized operations. This ensures that the uploaded global access secret in the console will not be used when OBS volumes are used. If you have not uploaded any AK/SK, skip this section.

To disable the global access secret, do as follows:

• Disable the automatic mounting of access secrets in the Everest add-on by setting disable_auto_mount_secret to true.

  The following steps apply when the add-on version is 2.x (for example, 2.1.42) or later.

  1. Log in to the CCE console and click the cluster name to access the cluster console.
  2. In the navigation pane, choose Add-ons. In the right pane, find the CCE Container Storage (Everest) add-on and click Edit.
  3. Configure the add-on parameters. Set Prohibit Global Secret from Mounting Object Storage (disable_auto_mount_secret) to Yes.
  4. Click OK.

  The following steps apply when the add-on version is 1.x. (The modified settings cannot be retained during the add-on upgrades. You are advised to use the add-on v2.x.)

  1. Use kubectl to access the cluster and run the following command to modify the add-on configuration:

     kubectl edit ds everest-csi-driver -nkube-system
     
     

  2. Search for disable-auto-mount-secret and set it to true.

     

  3. Run :wq to save the modification and exit. Wait until the pod is restarted.
• In the Settings > Cluster Settings area, disable the global access secret of the cluster. The global access secret (paas.longaksk) in the kube-system namespace of the cluster will be deleted.

1. Access the My Credentials page.
2. In the navigation pane, choose Access Keys.
3. Click Create Access Key. The Create Access Key dialog box is displayed.
4. Click OK to download the access key.

1. Obtain access keys.
2. Encode the access keys using Base64. (Assume that the AK is xxx and the SK is yyy.)

   echo -n xxx|base64
   echo -n yyy|base64
   
   

   Record the encoded AK and SK.

3. Create a YAML file for the secret, for example, test-user.yaml.

   apiVersion: v1
   data:
     access.key: WE5WWVhVNU*****
     secret.key: Nnk4emJyZ0*****
   kind: Secret
   metadata:
     name: test-user
     namespace: default
     labels:
       secret.kubernetes.io/used-by: csi
   type: cfe/secure-opaque
   
   

   Specifically:

   Parameter	Description
   access.key	A Base64-encoded AK
   secret.key	A Base64-encoded SK
   name	Secret name
   namespace	Namespace of the secret
   secret.kubernetes.io/used-by: csi	Add this label if you want to make it available on the CCE console when you create an OBS PV/PVC.
   type	Secret type. The value must be cfe/secure-opaque. When this type is used, the data entered by users is automatically encrypted.

4. Create the secret.

   kubectl create -f test-user.yaml
   
   

After a secret is created using the AK/SK, you can associate the secret with the PV to be created and then use the AK/SK in the secret to mount an OBS volume.

1. Log in to the OBS console, create an OBS bucket, and record the bucket name and StorageClass. The parallel file system is used as an example.
2. Create a YAML file for the PV, for example, pv-example.yaml.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv-obs-example
     annotations:
       pv.kubernetes.io/provisioned-by: everest-csi-provisioner
   spec:
     accessModes:
     - ReadWriteMany
     capacity:
       storage: 1Gi
     csi:
       nodePublishSecretRef:
         name: test-user
         namespace: default
       driver: obs.csi.everest.io
       fsType: obsfs
       volumeAttributes:
         everest.io/obs-volume-type: STANDARD
         everest.io/region: ru-moscow-1
         storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner
       volumeHandle: obs-normal-static-pv
     persistentVolumeReclaimPolicy: Delete
     storageClassName: csi-obs
   
   

   Parameter	Description
   nodePublishSecretRef	Secret specified during the mounting. name: name of the secret namespace: namespace of the secret
   fsType	File type, which can be s3fs or obsfs. If the value is s3fs, an OBS bucket is created. If the value is obsfs, an OBS parallel file system is created.
   volumeHandle	OBS volume name.

3. Create a PV.

   kubectl create -f pv-example.yaml
   
   

   After a PV is created, you can create a PVC and associate it with the PV.

4. Create a YAML file for the PVC, for example, pvc-example.yaml.

   Example YAML file for the PVC:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     annotations:
       csi.storage.k8s.io/node-publish-secret-name: test-user
       csi.storage.k8s.io/node-publish-secret-namespace: default
       volume.beta.kubernetes.io/storage-provisioner: everest-csi-provisioner
       everest.io/obs-volume-type: STANDARD
       csi.storage.k8s.io/fstype: obsfs
     name: obs-secret
     namespace: default
   spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 1Gi
     storageClassName: csi-obs
     volumeName: pv-obs-example
   
   

   Parameter	Description
   csi.storage.k8s.io/node-publish-secret-name	Secret name
   csi.storage.k8s.io/node-publish-secret-namespace	Namespace of the secret

5. Create a PVC.

   kubectl create -f pvc-example.yaml
   
   

   After the PVC is created, you can create a workload and associate it with the PVC to create volumes.

When dynamically creating an OBS volume, you can use the following method to specify a secret:

1. Create a YAML file for the PVC, for example, pvc-example.yaml.

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     annotations:
       csi.storage.k8s.io/node-publish-secret-name: test-user
       csi.storage.k8s.io/node-publish-secret-namespace: default
       everest.io/obs-volume-type: STANDARD
       csi.storage.k8s.io/fstype: obsfs
     name: obs-secret
     namespace: default
   spec:
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 1Gi
     storageClassName: csi-obs
   
   

   Parameter	Description
   csi.storage.k8s.io/node-publish-secret-name	Secret name
   csi.storage.k8s.io/node-publish-secret-namespace	Namespace of the secret

2. Create a PVC.

   kubectl create -f pvc-example.yaml
   
   

   After the PVC is created, you can create a workload and associate it with the PVC to create volumes.

You can use a secret of an IAM user to mount an OBS volume. Assume that a workload named obs-secret is created, the mount path in the container is /temp, and the IAM user has the CCE ReadOnlyAccess and Tenant Guest permissions.

1. Query the name of the workload pod.

   kubectl get pod | grep obs-secret
   
   

   Expected outputs:

   obs-secret-5cd558f76f-vxslv          1/1     Running   0          3m22s
   
   

2. Query the objects in the mount path. In this example, the query is successful.

   kubectl exec obs-secret-5cd558f76f-vxslv -- ls -l /temp/
   
   

3. Write data into the mount path. In this example, the write operation failed.

   kubectl exec obs-secret-5cd558f76f-vxslv -- touch /temp/test
   
   

   Expected outputs:

   touch: setting times of '/temp/test': No such file or directory
   command terminated with exit code 1
   
   

4. Set the read/write permissions for the IAM user who mounted the OBS volume by referring to the bucket policy configuration.

5. Write data into the mount path again. In this example, the write operation succeeded.

   kubectl exec obs-secret-5cd558f76f-vxslv -- touch /temp/test
   
   

6. Check the mount path in the container to see whether the data is successfully written.

   kubectl exec obs-secret-5cd558f76f-vxslv -- ls -l /temp/
   
   

   Expected outputs:

   -rwxrwxrwx 1 root root 0 Jun  7 01:52 test