Querying and Managing Elasticsearch Cluster Logs

Prerequisites

• Storage Class: Standard.
• Region: the same as that of the cluster.

Querying Logs

1. Log in to the CSS management console.
2. Choose Clusters in the navigation pane. On the Clusters page, click the cluster whose logs you want to query. The cluster information page is displayed.
3. In the navigation pane on the left, choose Log Management.
4. Query logs on the log management page.
5. Select the node, log type, and log level, and click . The query result is displayed.

   The latest 10,000 logs can be queried, and a maximum of 100 records are displayed.

   When a log file reaches 128 MB, the system automatically compresses and archives it. Only unarchived logs appear on the log query page, while archived logs remain accessible through the log backup function.

Backing Up Logs

1. Log in to the CSS management console.
2. Choose Clusters in the navigation pane. On the Clusters page, click the name of the target cluster. The cluster information page is displayed.
3. Click the Logs tab and toggle on the Log Management switch.
4. In the Edit Log Backup Configuration dialog box, set the parameters.

   If log backup has been enabled for the cluster, you can also click the edit icon next to Log Backup Configuration to update the settings.

   Table 1 Log backup settings

   Parameter	Description
   OBS Bucket	Select an OBS bucket for storing log backups from the drop-down list box. If no OBS buckets meet your requirements, click Create Bucket to go to the OBS console and create one. Select an OBS bucket that meets the following requirements: Storage Class: Standard. Region: the same as that of the cluster. To grant an IAM user access to an OBS bucket, you need to grant the GetBucketStoragePolicy, GetBucketLocation, ListBucket, and ListAllMyBuckets permissions to that user.
   Backup Path	Set the log storage location in the OBS bucket. The backup path cannot: Contain the following characters: \:*?"<>|'{} Start with a slash (/). Start or end with a period (.). Contain more than two consecutive slashes (/) or periods (.). Exceed 512 characters.
   IAM Agency	Select an IAM agency to grant the current account the permission to access and use OBS. To store logs to an OBS bucket, you must have the required OBS access permissions. The IAM agency must meet the following requirements: Agency Type must be Cloud service. Set Cloud Service to Elasticsearch or CSS. Mandatory policies: Tenant Administrator If no agency is available, contact the CSS administrator to create one. WARNING: The agency name can contain only letters (case-sensitive), digits, underscores (_), and hyphens (-). Otherwise, the backup will fail.

5. Back up logs.
   • Automatically backing up logs

     Click the icon on the right of Auto Backup to enable automatic log backup.

     After automatic backup is enabled, set the backup start time in the Configure Auto Backup dialog box. When the scheduled time arrives, the system will back up logs automatically.

     After Automatic Snapshot Creation is enabled, you can click the Edit icon on the right of the parameter to change the backup start time.

   • Manually backing up logs

     On the Log Backup tab page, click Back Up. In the displayed dialog box, click Yes to start backup.

     If Task Status in the log backup list is Successful, the backup is successful.

6. View log files.

   Logs are backed up incrementally. After the backup is successful, you can access the target OBS bucket to obtain the full log files. Table 2 lists the log types, where clustername indicates the cluster name.

   Table 2 Log types

   Log Name	Description
   clustername_deprecation.log	Deprecation log file
   clustername_index_indexing_slowlog.log	Slow indexing log file
   clustername_index_search_slowlog.log	Slow query log file
   clustername.log	Run log file
   clustername_access.log	Access log file

Log Introduction

• Run log description

  Run logs record the cluster status and key information about write and query operations. For example, the log record below indicates that an index named test was created and afterwards the cluster status changed from YELLOW to GREEN.

  Figure 1 A sample of run logs

  
  

  Log content:

  • 1. Log generation time
  • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
  • 3. Log-generating module
  • 4. Name of the log-generating node
  • 5. Log content

• Slow indexing log description

  Slow indexing logs record indexing operations that took a long time to complete. For example, the log record below shows an indexing request that lasted longer than the preset threshold. The log contains the index name, duration, and request content.

  Figure 2 A sample of slow indexing logs

  
  

  Log content:

  • 1. Log generation time

  • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
  • 3. Log-generating module
  • 4. Name of the log-generating node
  • 5. Index name and ID
  • 6. Log content In this example, the log recorded the request execution duration, index type, and index request body.

• Slow query log description

  Slow query logs record search requests that took a long time to complete. For example, the log record below shows a search request that lasted longer than the preset threshold. The log contains the index name, duration, and request content.

  Figure 3 A sample of slow query logs

  
  

  Log content:

  • 1. Log generation time
  • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
  • 3. Log-generating module
  • 4. Name of the log-generating node
  • 5. Index name and shard ID
  • 6. Log content In this example, the log recorded the query duration, number of hits, and query request body.

• Deprecation log description

  Deprecation logs record deprecation warnings. For example, the log record below indicates that GET /_cat/master has been deprecated and should be replaced with GET /_cat/cluster_manager.

  Figure 4 A sample of deprecation logs

  
  

  Log content:

  • 1. Log generation time
  • 2. Log level, which can be DEBUG, INFO, WARN, ERROR, or DEPRECATION
  • 3. Log-generating module
  • 4. Name of the log-generating node
  • 5. Log content

• Access log description

  Access logs record cluster access requests and source addresses. For example, the log record below has recorded source information for the /_snapshot/my_backup/my_snapshot/_restore?pretty=true operation.

  Figure 5 A sample of access logs

  
  

  Log content:

  • 1. Log generation time
  • 2. Name of the log-generating node
  • 3. Name of the log-generating thread
  • 4. Log level, which can be DEBUG, INFO, WARN, or ERROR
  • 5. Log request method
  • 6. Request path
  • 7. Source and destination addresses of the request

Related Documents