Creating a Snapshot to Back Up the Data of an Elasticsearch Cluster

Constraints

• When creating a backup for the first time, you are advised to back up all indexes.
• Cluster snapshots will increase CPU usage and disk I/O. You are advised to take cluster snapshots during off-peak hours.
• After the OBS bucket that stores a snapshot is changed, the cluster obtains snapshot data from the new bucket.
• If a cluster is in the Unavailable state, you can use the cluster snapshot function only to restore the cluster or view snapshot information.
• While you are creating a snapshot for a cluster or restoring it using one, you can perform certain operations, including scaling out the cluster (except when the cluster is the destination of a restoration operation), accessing Kibana, viewing metrics, and deleting other snapshots. However, you cannot perform the following operations: restarting or deleting the cluster, deleting a snapshot that is in the Creating or Restoring state, and creating or restoring another snapshot. While a snapshot is being created or restored for a cluster, any automatic snapshot creation task initiated for that cluster will be canceled.
• The first snapshot of a cluster is a full snapshot, and subsequent snapshots are incremental snapshots. CSS snapshot files depend on each other.

Prerequisites

• You have created an OBS bucket for storing snapshots. Storing snapshots in an OBS bucket may incur additional fees. For details, see section "Billing Overview" in OBS User Guide.
• You have the permission to create and restore snapshots. On the IAM console, choose Permissions > Roles to check whether the account you are using has both of the following permissions. If the answer is no, contact the service administrator to add them.
  • Elasticsearch Administrator or CSS Administrator
  • Tenant Administrator or OBS Administrator

Setting Automatic Snapshot Creation

1. Log in to the CSS management console.
2. Enable automatic snapshot creation for a cluster. You can enable automatic snapshot creation when creating a new cluster. For details, see 7. Alternatively, you can do that for an existing cluster.
   • If automatic snapshot creation has been enabled during cluster creation, go to 7.
   • To enable automatic snapshot creation after a cluster is already created, go to the next step.
3. On the Clusters page, click the name of the target cluster. In the navigation pane on the left, choose Cluster Snapshots.
4. On the displayed Cluster Snapshots page, click the icon to the right of Cluster Snapshot to enable the cluster snapshot function. If it is already enabled, skip this step.

   CSS will automatically create an OBS bucket and IAM agency for snapshot storage. The automatically created OBS bucket and IAM agency will be displayed on the page. You can also click on the right of Basic Configuration to edit the configuration.

   Table 1 Basic configuration for a cluster snapshot policy

   Parameter	Description
   OBS Bucket	Select an OBS bucket for storing snapshots from the drop-down list box. You can also click Create Bucket on the right to create an OBS bucket. The created or existing OBS bucket must meet the following requirements: Storage Class is Standard. Region must be the same as that of the created cluster.
   Backup Path	Storage path of the snapshot in the OBS bucket. The backup path cannot: Contain the following characters: \:*?"<>|'{} Start with a slash (/). Start or end with a period (.). Contain more than two consecutive slashes (/) or periods (.). Exceed 512 characters.
   IAM Agency	IAM agency authorized by the current account for CSS to access or maintain data stored in OBS. If no agency is available, contact the CSS administrator to create one. The selected IAM agency must meet the following requirements: Agency Type must be Cloud service. Set Cloud Service to Elasticsearch or CSS. Mandatory policies: Tenant Administrator or OBS Administrator. NOTE: The agency name can contain only letters (case-sensitive), digits, underscores (_), and hyphens (-). Otherwise, the backup will fail.

5. Enable the automatic snapshot creation function. The Configure Automatic Snapshot Creation dialog box is displayed. If the automatic snapshot creation function is enabled, you can click on the right of Automatic Snapshot Creation to modify the snapshot policy.

   Table 2 Setting automatic snapshot creation

   Parameter	Description
   Snapshot Name Prefix	The snapshot name prefix contains 1 to 32 characters and must start with a lowercase letter. Only lowercase letters, digits, hyphens (-), and underscores (_) are allowed. A snapshot name consists of a snapshot name prefix and a timestamp, for example, snapshot-1566921603720.
   Time Zone	Time zone for the backup time, which cannot be changed. Specify Backup Started Time based on the time zone.
   Backup Start Time	The time when the backup starts automatically every day. You can specify this parameter only in full hours, for example, 00:00 or 01:00. The value ranges from 00:00 to 23:00. Select a time from the drop-down list.
   Index	Indexes that you want to back up using snapshots. The index names cannot contain spaces or uppercase letters, and cannot contain "\<|>/?. Use commas (,) to separate different index names. If you do not specify this parameter, all indexes in the cluster are backed up by default. You can use the asterisk (*) to match multiple indexes. For example, if you enter index*, then the data of all indexes whose names are prefixed with index will be backed up. NOTE: Run the GET /_cat/indices command in Kibana to query the names of all indexes in the cluster.

6. Click OK to save the snapshot policy.
7. Snapshots that are automatically created according to the snapshot policy are displayed in the snapshot list, along with manually created snapshots. You can distinguish between them based on Snapshot Type. In the upper right corner of the snapshot list, enter the keyword of the snapshot name or snapshot ID to search for the desired snapshots.
8. (Optional) Disable the automatic snapshot creation function.

   After you disable the automatic snapshot creation function, the system stops automatic creation of snapshots. If the system is creating a snapshot based on the automatic snapshot creation policy and the snapshot is not yet displayed in the snapshot list, you cannot disable the automatic snapshot creation function. In this case, if you click the button next to Automatic Snapshot Creation, a message is displayed, indicating that you cannot disable the function. You are advised to disable the function after the system completes automatic creation of the snapshot, and the created snapshot is displayed in the snapshot list.

   When disabling the automatic snapshot creation function, you can choose whether to delete the snapshots that have been automatically created by selecting Delete automated snapshots in the displayed dialog box. By default, automatically created snapshots are not deleted.

   • If you do not select Delete automated snapshots, automatically created snapshots are not deleted when you disable the automatic snapshot creation function. You can manually delete them later. For details, see Deleting an Elasticsearch Cluster Snapshot. If you do not manually delete the automatically created snapshots and enable the automatic snapshot creation function again, then all snapshots with Snapshot Type set to Automated in the snapshot list of the cluster can only be automatically deleted by the system. The system automatically deletes snapshots based on the policy configured when the automatic snapshot creation function is enabled. For example, if the number of retained snapshots is set to 10 in this policy and more than 10 snapshots are created, the system automatically deletes the excess snapshots on the half hour.
   • If you select Delete automated snapshots, all snapshots with Snapshot Type set to Automated in the snapshot list will be deleted when you disable the automatic snapshot creation function.
   Note

   If snapshots are disabled, existing snapshots will not be automatically deleted. If you need to delete the snapshots, manage the bucket that stores snapshots on the OBS console.

Manually Creating a Snapshot

1. Log in to the CSS management console.
2. On the Clusters page, click the name of the target cluster. In the navigation pane on the left, choose Cluster Snapshots.
3. On the displayed Cluster Snapshots page, click the icon to the right of Cluster Snapshot to enable the cluster snapshot function. If it is already enabled, skip this step.

   CSS will automatically create an OBS bucket and IAM agency for snapshot storage. The automatically created OBS bucket and IAM agency will be displayed on the page. You can also click on the right of Basic Configuration to edit the configuration.

   Table 3 Basic configuration for a cluster snapshot policy

   Parameter	Description
   OBS Bucket	Select an OBS bucket for storing snapshots from the drop-down list box. You can also click Create Bucket on the right to create an OBS bucket. The created or existing OBS bucket must meet the following requirements: Storage Class is Standard. Region must be the same as that of the created cluster.
   Backup Path	Storage path of the snapshot in the OBS bucket. The backup path cannot: Contain the following characters: \:*?"<>|'{} Start with a slash (/). Start or end with a period (.). Contain more than two consecutive slashes (/) or periods (.). Exceed 512 characters.
   IAM Agency	IAM agency authorized by the current account for CSS to access or maintain data stored in OBS. If no agency is available, contact the CSS administrator to create one. The selected IAM agency must meet the following requirements: Agency Type must be Cloud service. Set Cloud Service to Elasticsearch or CSS. Mandatory policies: Tenant Administrator or OBS Administrator. NOTE: The agency name can contain only letters (case-sensitive), digits, underscores (_), and hyphens (-). Otherwise, the backup will fail.

4. After basic configurations are completed, click Create to manually create a snapshot.

   Table 4 Parameters for manually creating a snapshot

   Parameter	Description
   Snapshot Name	Name of a manually created snapshot. It can contain 4 to 64 characters and must start with a lowercase letter. Only lowercase letters, digits, hyphens (-), and underscores (_) are allowed. For snapshots you create manually, you can specify the snapshot name. The system will not automatically add a timestamp to the snapshot name.
   Index	Indexes that you want to back up using snapshots. The index names cannot contain spaces or uppercase letters, and cannot contain "\<|>/?. Use commas (,) to separate different index names. If you do not specify this parameter, all indexes in the cluster are backed up by default. You can use the asterisk (*) to match multiple indexes. For example, if you enter index*, then the data of all indexes whose names are prefixed with index will be backed up. NOTE: Run the GET /_cat/indices command in Kibana to query the names of all indexes in the cluster.
   Description	Description of the snapshot. The value contains 0 to 256 characters and cannot contain <>.

5. Click OK to create the snapshot.

   After the snapshot is created, it is displayed in the snapshot list. The status Available indicates that the snapshot is created successfully. along with manually created snapshots. You can distinguish them by the Snapshot Type setting. In the upper right corner of the snapshot list, enter the keyword of the snapshot name or snapshot ID to search for the desired snapshots.