Облачная платформаAdvanced

Creating a Cluster

Язык статьи: Английский
Перевести

This topic describes how to create an OpenSearch cluster. You can create an OpenSearch cluster in either of the following ways:

  • Method 1: Create an OpenSearch cluster on the CSS management console.
  • Method 2: Create an OpenSearch cluster using CSS APIs. For details, see section "Creating a Cluster" in Cloud Search Service API Reference.

Scenarios

Table 1 lists key parameters that differentiate between different types of clusters.

Table 1 Parameters that differentiate between different types of clusters

Cluster Type

Security Mode

HTTPS Access

Internet Access

OpenSearch Dashboards Public Network Access

Cluster with the security mode disabled

Disabled

N/A

Cannot be enabled

Cannot be enabled

Cluster in security mode + HTTP

Enabled

Disabled

Cannot be enabled

Can be enabled

Cluster in security mode + HTTPS

Enabled

Enabled

Can be enabled

Can be enabled

Prerequisites

You have planned the OpenSearch cluster configuration by following the instructions in Cluster Planning.

Creating a Cluster

  1. Log in to the CSS management console.
  2. On the Dashboard page, click Create Cluster in the upper right corner. The Create Cluster page is displayed.

    Alternatively, choose Clusters > OpenSearch in the navigation tree on the left. Click Create Cluster in the upper right corner. The Create Cluster page is displayed.

  3. On the Basic Configuration page, configure basic information and resources for the OpenSearch cluster.
    Table 2 Basic settings of the OpenSearch cluster

    Parameter

    Description

    Region

    Select the region where the cluster is located.

    Resources in different regions cannot communicate with each other over an internal network. For lower network latency and quicker resource access, select the nearest region.

    AZ

    Select one or more AZs associated with the cluster region.

    A maximum of three AZs can be configured. For details about the use of multiple AZs, see Suggestions on Multi-AZ Deployment.

    Type

    Choose OpenSearch.

    Version

    Select a cluster version from the drop-down list.

    Name

    User-defined cluster name.

    The cluster name must start with a letter and can contain 4 to 32 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

    Cluster Description

    Add a description for the cluster for easy recognition.

    The value can contain 0 to 128 characters.

    Nodes

    Number of nodes in the cluster. Data nodes store the cluster's data. If there are no client nodes in the cluster, data nodes will need to additionally handle cluster access and data analytics requests; if there are no master nodes, data nodes will also need to provide cluster management.

    • If master nodes are configured, the number of data nodes ranges from 1 to 200.
    • If no master nodes are configured, the number of data nodes ranges from 1 to 32.

    Principles:

    • To ensure high service availability and prevent individual node overload, a single-AZ cluster should have at least three data nodes; a two-AZ cluster must have at least four data nodes (two in each AZ); and a three-AZ cluster must have at least three data nodes (one in each AZ).
    • For optimal performance, the number of data nodes should be an integer multiple of the number of AZs, for example, six data nodes for a three-AZ cluster. This helps prevent uneven data distribution and ensure optimal query and ingestion performance.

    CPU Architecture

    x86. The supported types depend on the actual regional environment.

    Node Specifications

    Data node flavor. You can select a node flavor based on your needs. Each cluster supports only one flavor.

    Node Storage Type

    If you select EVS for node storage, you need to further select the EVS disk type for data nodes of the cluster. Supported node storage types: Common I/O, High I/O, Ultra-high I/O.

    Node Storage Capacity

    Data node storage capacity. Its value range varies with node specifications.

    The node storage capacity must be a multiple of 20.

    Master node

    The master node is responsible for important cluster management tasks, such as metadata management, index creation and deletion, and shard allocation. It plays a critical role in metadata management, node management, stability guarantee, and cluster operation control for large-scale clusters.

    After enabling the master node, specify Node Specifications, Nodes, and Node Storage Type. The value of Nodes must be an odd number greater than or equal to 3. Up to nine nodes are supported. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.

    Client node

    Client nodes receive and coordinate external requests, such as search and write requests. They play an important role in handling high-load queries, complex aggregations, managing a large number of shards, and improving cluster scalability.

    After enabling the client node, specify Node Specifications, Nodes and Node Storage Type. The value of Nodes ranges from 1 to 64. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.

    To ensure high service availability and prevent individual node overload in a multi-AZ deployment, the number of client nodes should be greater than or equal to the number of AZs. (For a cluster spanning two AZs, this means a minimum of two client nodes; for a three-AZ deployment, a minimum of three client nodes.)

    Cold data node

    Cold data nodes are used to store query latency-insensitive data in large quantities. They offer an effective way to manage large datasets and cut storage costs.

    After enabling cold data nodes, configure Node Specifications, Nodes, Node Storage Type, and Node Storage Capacity. The value of Nodes ranges from 1 to 32. Select Node Storage Type and Node Storage Capacity as required.

    • To ensure high service availability and prevent individual node overload, a two-AZ cluster must have at least four cold data nodes (two in each AZ); and a three-AZ cluster must have at least three cold data nodes (one in each AZ).
    • To prevent uneven data distribution and ensure optimal query and ingestion performance, the number of cold data nodes should be an integer multiple of the number of AZs, for example, six cold data nodes for a three-AZ cluster.

    When cold data nodes are enabled, you can switch between cold and hot data nodes. For details, see Switching Between Hot and Cold Data.

    Enterprise Project

    When creating a CSS cluster, you can bind an enterprise project to the cluster if you have enabled the enterprise project function.

    Select an enterprise project from the Enterprise Project drop-down list, or click View Enterprise Project to go to the Enterprise Project Management Service page and check existing enterprise projects.

  4. Click Next: Network.
  5. On the Network page, configure the network settings and security mode for the OpenSearch cluster.
    Table 3 Network settings for the OpenSearch cluster

    Parameter

    Description

    VPC

    Specify a VPC to isolate the cluster's network.

    Click View VPC to go to the VPC management console and check the created VPCs.

    If no VPC is available, contact the CSS administrator to create a new VPC. For details, see section "Creating a VPC with a Subnet" in VPC User Guide.

    NOTE:

    The VPC must contain CIDRs. Otherwise, cluster creation will fail. By default, a created VPC contains CIDRs.

    Subnet

    A subnet provides dedicated network resources that are isolated from other networks, improving network security.

    Select a subnet needed by the cluster in the current VPC.

    If no existing subnets meet your requirements, click Create Subnet to create a new subnet. For details, see section "Creating a VPC with a Subnet" in Virtual Private Cloud User Guide.

    Security Group

    A security group serves as a virtual firewall that provides access control policies for clusters.

    Select a security group for the cluster. Click View Security Group to go to the security group list, where you can view details about security groups.

    NOTE:

    Ensure that Port Range/ICMP Type is set to a port range that includes port 9200 for the selected security group.

    Security Mode

    Whether to enable the security mode for the cluster.

    • The security mode is enabled by default. In security mode, a cluster's communication is encrypted and access to the cluster requires user authentication. This is why Administrator Username and Administrator Password must be configured for the cluster.
      • The default administrator username is admin.
      • Set and confirm the Administrator Password. This password will be required when you access this cluster.
    • If Security Mode is disabled, a cluster in non-security mode will be created. With such a cluster, access to the cluster will require no user authentication, and data will be transmitted in plaintext using HTTP. Make sure the cluster is deployed in a secure environment. Do not expose the cluster's network interface to the public network.

    HTTPS Access

    HTTPS access can be enabled only when security mode is enabled for the cluster. With HTTPS access enabled, communication will be encrypted when you access the cluster.

    HTTPS can be disabled only for OpenSearch 1.3.6, 2.19.0, and 3.4.0 clusters for which the security mode is enabled. For other versions, HTTPS access cannot be disabled.

    NOTE:

    Compared with a non-security mode cluster that uses HTTP, a security-mode cluster that uses HTTPS has lower read performance. The performance loss is estimated at around 20% under high concurrency. If you want fast read performance as well as the isolation and permission control (such as indexes, documents, and fields) enabled by the security mode, you can disable HTTPS Access. After HTTPS Access is disabled, HTTP protocol is used for cluster communication. In this case, data security cannot be guaranteed and public network access is disabled.

    Public IP Address

    This parameter is available only when Security Mode and HTTPS Access are enabled. When Public IP Address is enabled, a public IP address is automatically assigned, which will enable access to the security cluster from the Internet. For details, see Configuring Public Network Access.

  6. Click Next: Advanced Settings.
  7. On the Advanced Settings page, configure a snapshot policy and other advanced settings for the OpenSearch cluster.
    1. Set a cluster snapshot policy.

      Cluster snapshots are disabled by default. To enable them, toggle on Cluster Snapshots. To store automatically created snapshots in OBS, an agency will need to be created in order to access OBS. Fees will be incurred for using OBS storage (storage class: Standard).

      Table 4 Basic configuration for a cluster snapshot policy

      Parameter

      Description

      OBS Bucket

      From the drop-down list, select an OBS bucket for storing snapshots. You can also click Create Bucket on the right to create a new OBS bucket.

      The newly created or existing OBS bucket must meet the following requirements:

      • Storage Class: Standard.
      • Region must be the same as that of the created cluster.

      Backup Path

      Snapshot storage path in the OBS bucket.

      The backup path cannot:

      • Contain the following characters: \:*?"<>|'{}
      • Start with a slash (/).
      • Start or end with a period (.).
      • Contain more than two consecutive slashes (/) or periods (.).
      • Exceed 512 characters.

      CAUTION:

      Only standard OBS storage can be used to store snapshots. Do not apply OBS lifecycle rules to the backup path.

      Maximum Backup Rate (per Second)

      The parameter sets the maximum backup rate per node. When it is exceeded, flow control is triggered to prevent excessive resource usage and ensure system stability. The actual backup rate may not reach the configured value, as it depends on factors such as OBS performance and disk I/O.

      Value format: number + unit

      • Number range: 0–9999
      • Unit: KB, MB, GB, TB, PB, or B

      Default value: 40 MB

      The value 0MB means there is no limit on how fast data is backed up to snapshots. An overly high backup rate may lead to excessive resource usage, which may impact cluster stability. Configure this parameter carefully to maintain optimal performance.

      Maximum Recovery Rate (per Second)

      The parameter sets the maximum recovery rate per node. When it is exceeded, flow control is triggered to prevent excessive resource usage and ensure system stability. The actual recovery rate may not reach the configured value, as it depends on factors such as OBS performance and disk I/O.

      Value format: number + unit

      • Number range: 0–9999
      • Unit: KB, MB, GB, TB, PB, or B

      The default value is 0MB, indicating no limit.

      An overly high recovery rate may lead to excessive resource usage, which may impact cluster stability. Configure this parameter carefully to maintain optimal performance.

      For OpenSearch clusters, the recovery rate is also limited by the indices.recovery.max_bytes_per_sec parameter.

      • If Maximum Recovery Rate (per Second) is less than indices.recovery.max_bytes_per_sec, the former takes effect.
      • If Maximum Recovery Rate (per Second) is greater than indices.recovery.max_bytes_per_sec, the latter takes effect.

      NOTE:
      • To check the value of indices.recovery.max_bytes_per_sec, run the following command:
        GET _cluster/settings
      • To modify indices.recovery.max_bytes_per_sec, run the following command:
        PUT _cluster/settings
        {
        "transient": {
        "indices.recovery.max_bytes_per_sec": "100mb"
        }
        }

      IAM Agency

      IAM agency authorized by the current account for CSS to access or maintain data stored in OBS. If no agency is available, contact the CSS administrator to create one.

      The selected IAM agency must meet the following requirements:

      • Agency Type must be Cloud service.
      • Set Cloud Service to Elasticsearch or CSS.
      • Mandatory policies: Tenant Administrator or OBS Administrator.

      WARNING:

      The agency name can contain only letters (case-sensitive), digits, underscores (_), and hyphens (-). Otherwise, the backup will fail.

      Table 5 Setting automatic snapshot creation

      Parameter

      Description

      Snapshot Name Prefix

      The snapshot name prefix contains 1 to 32 characters and must start with a lowercase letter. Only lowercase letters, digits, hyphens (-), and underscores (_) are allowed. A snapshot name consists of a snapshot name prefix and a timestamp, for example, snapshot-1566921603720.

      Time Zone

      Time zone for the backup time. Specify Backup Started Time based on the time zone.

      Backup Start Time

      Specify the start time of auto backup.

      Select a value from the drop-down list. The interval can be Daily, Hourly, or weekly (by selecting a specific day of the week), and the backup time can be set to any hour from 00:00 to 23:00 (full hours only).

    2. Configure advanced settings for the cluster.

      Tags

      Adding tags to clusters helps you identify and manage your cluster resources. You can customize tags or use tags predefined by Tag Management Service (TMS). If you want to use the same tag to identify multiple cloud resources for better resource grouping, we recommend that you predefine tags in Tag Management Service (TMS). For details, see Tag Management Service User Guide.

      Note

      If your organization has configured tag policies for CSS, add tags to clusters based on these policies. Tags that do not comply with predefined tag policies will cause a cluster creation failure. Contact the administrator to learn about the tag policies.

  8. Click Next: Confirm Configuration. Check the configuration and click Next to create a cluster.
  9. Return to the cluster list and check the newly created cluster. If the cluster is created successfully, Cluster Status changes to Available. Cluster creation time depends on the number of nodes. Typically, this process takes less than 60 minutes, though clusters with a large number of nodes may require additional time.

    If cluster creation fails, try creating the cluster again by rectifying the errors returned.