Configuring Index State Management (ISM) Policies

• Only Elasticsearch clusters 7.6.2 or later support the ISM plugin.
• The Kibana GUI varies depending on the Kibana version. Kibana 7.10.2 is used as an example here.

Define one or more ISM policies on Kibana to tell the cluster how to manage indexes.

1. Log in to the CSS management console.
2. In the navigation pane on the left, choose Clusters > Elasticsearch.
3. In the cluster list, find the target cluster, and click Kibana in the Operation column to log in to the Kibana console.
4. On the Kibana console, expand the menu in the upper-left corner, and choose IM or Index Management to go to the index management page.
5. Select Index Policies.
6. Click Create policy to configure an index lifecycle policy.
   • Policy ID: Enter a custom policy name.
   • Define policy: Define the policy in JSON format. You can define a policy by modifying the default system example.

   Figure 1 Creating a policy

   
   
7. Click Create.

After an ISM policy is created, associate it with specific indexes to apply it. You are advised to associate a policy with an index template. This ensures that the policy is automatically applied to all indices created based on this template.

• Method 1: Associate a policy with an index template (recommended)

  Use this method for indexes that store time series data.

  1. In the left navigation pane on Kibana, choose Dev Tools.

     The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.

  2. Run the following command to create or update an index template:

     PUT _template/<template_name>   
     {
       "index_patterns": ["index_name-*"],  // Match all indexes whose name starts with index_name-.
       "settings": {
         "opendistro.index_state_management.policy_id": "Policy_ID" // Replace it with the target policy ID.
       }
     }
     
     

     For more information, see Create template.

  3. On the Managed Indices page, check for newly-created indexes whose name starts with index_name-*, and check whether their Policy is the associated policy ID.
• Method 2: Manually associating an existing index with a policy

  Use this method to associate a policy with existing indexes that are currently not associated with any policy.

  1. On the Kibana console, expand the menu in the upper-left corner, and choose IM or Index Management to go to the index management page.
  2. Click Indices.
  3. In the Indices list, select one or more indexes.
  4. Click Apply policy in the upper right corner. In the displayed dialog box, select a policy ID.

     Figure 2 Apply policy

     
     
  5. Click Apply.
  6. On the Managed Indices page, locate the indexes and make sure their Policy is the associated policy ID.

After you associate a policy with indexes, ISM automatically creates a job that runs every 5 minutes to execute the policy, check criteria, and change index states based on the policy.

To manage deployed ISM policies, choose Index Management > Managed Indices on Kibana.

• View status: In the Managed Indices list, check the policy associated with each index and the policy status.
• Retry policy: If the policy associated with an index is not working properly, select this index and click Retry policy.
• Change policy: To update the policy for an index, click Change policy.
• Remove policy: To stop lifecycle management for an index, click Remove policy.

A lifecycle policy can be configured to enable automatic index rollover, that is, automatically creating a new index when an existing index meets certain conditions (for example, when it reaches a predefined age in terms of days). The new index then receives subsequent writes. By default, even if an existing index contains no documents, new indexes are still created. Over time, this can result in a large number of empty indexes. To prevent this, CSS allows you to disable automatic rollover for empty indexes. This is achieved by configuring a lifecycle policy to allow automatic rollover only for indexes that contain documents. By doing so, you can prevent excessive empty indexes from being created, thus optimizing storage utilization and enhancing overall system efficiency.

This feature is supported only when an Elasticsearch cluster meets the following conditions:

• Cluster version: 7.6.2 or 7.10.2
• Image version: x.x.x_25.9.0_xxx or later

You can disable automatic rollover for empty indexes at either the index level or the cluster level. Index-level settings take precedence over cluster-level settings. You must configure this manually. The commands are as follows:

• Disable empty index automatic rollover for a specified index:

  PUT {index_name}/_settings
  {
    "index.plugins.index_state_management.rollover.only_if_has_documents": true
  }
  
  

• Disable empty index automatic rollover at the cluster level:

  PUT _cluster/settings
  {
    "persistent": {
      "plugins.index_state_management.rollover.only_if_has_documents": true
    }
  }
  
  

ISM tasks (such as converting hot data to cold data and deleting expired indexes) can fail due to transient cluster issues like resource constraints, node restarts, or network interruptions. To address this, CSS automatically retries (reactivates) failed ISM tasks at configured intervals until they succeed. This ensures the continuity and reliability of ISM tasks.

This feature is supported only when an Elasticsearch cluster meets the following conditions:

• Cluster version: 7.6.2 or 7.10.2
• Image version: x.x.x_25.9.0_xxx or later

Automatic retry of ISM tasks is enabled by default for Elasticsearch clusters that meet the conditions above.

For Elasticsearch clusters that do not meet these conditions, upgrade them to enable this feature. However, ISM tasks that have already failed before the upgrade will not be retried after the cluster is upgraded. You will need to manually retry them before the automatic retry can occur. The command for retrying failed ISM tasks is as follows:

Run the following command to modify automatic retry settings for index lifecycle management tasks:

PUT _cluster/settings
{
  "persistent": {
    "plugins.index_state_management.coordinator.css.reactivate": true,
    "plugins.index_state_management.coordinator.css.reactivate_period": "5m",
    "plugins.index_state_management.coordinator.css.reactivate_duration": "30m",
    "plugins.index_state_management.coordinator.css.reactivate_max_duration": "12h",
    "plugins.index_state_management.coordinator.css.max_inflight_reactivate_tasks": 1000
  }
}

Idempotence considerations for ISM task retry

• Idempotent operations: All steps are retried, starting from the beginning, to ensure data consistency.
• Non-idempotent operations: The retry begins from where the failure occurred. If the original failure cause persists, retries may fail repeatedly, in which case, manual intervention is required.

  Examples of non-idempotent operations in Elasticsearch's index lifecycle management tasks: ForceMerge, Notification, and Snapshot.

In the ISM task details API, CSS has added fields that record ISM task execution details. You can run the following command to check ISM task records, including the total number of failures, failure occurrence time, activation time, and failure causes:

Example response: