Configuring LoadBalancer Services Using Annotations
You can add annotations to a YAML file to use some CCE advanced functions. This section describes the available annotations when a LoadBalancer service is created.
- Interconnection with ELB
- Sticky Session
- Health Check
- HTTP or HTTPS
- SNI
- Dynamic Adjustment of the Weight of the Backend ECS
- Passthrough Capability
- Blocklist/Trustlist
- Host Network
- Timeout
- Resource Tags
- HTTP/2
- Obtaining Client IP Addresses
- Changing a Custom EIP
- Configuring Port Ranges for a Listener
Interconnection with ELB
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.class | String | Select a proper load balancer type. Options:
The default value is union. | v1.9 or later |
kubernetes.io/elb.id | String | Mandatory when an existing load balancer is associated. ID of a load balancer. How to obtain: On the management console, click Service List, and choose Networking > Elastic Load Balance. Click the name of the target load balancer. On the Summary tab page, find and copy the ID. NOTE: The system preferentially connects to the load balancer based on the kubernetes.io/elb.id field. If this field is not specified, the spec.loadBalancerIP field is used (optional and available only in 1.23 and earlier versions). Do not use the spec.loadBalancerIP field to connect to the load balancer. This field will be discarded by Kubernetes. For details, see Deprecation. | v1.9 or later |
kubernetes.io/elb.autocreate | Mandatory when load balancers are automatically created. Example
| v1.9 or later | |
kubernetes.io/elb.enterpriseID | String | Optional when load balancers are automatically created. Clusters of v1.15 and later versions support this field. In clusters earlier than v1.15, load balancers are created in the default project by default. This parameter indicates the ID of the enterprise project in which the ELB load balancer will be created. If this parameter is not specified or is set to 0, resources will be bound to the default enterprise project. How to obtain: Log in to the management console and choose Enterprise > Project Management on the top menu bar. In the list displayed, click the name of the target enterprise project, and copy the ID on the enterprise project details page. | v1.15 or later |
kubernetes.io/elb.subnet-id | String | Optional when load balancers are automatically created. ID of the subnet where the cluster is located. The value can contain 1 to 100 characters.
| Mandatory for clusters of a version earlier than v1.11.7-r0 Discarded in clusters of a version later than v1.11.7-r0 |
kubernetes.io/elb.lb-algorithm | String | Specifies the load balancing algorithm of the backend server group. The default value is ROUND_ROBIN. Options:
NOTE: If this parameter is set to SOURCE_IP, the weight setting (weight field) of backend servers bound to the backend server group is invalid, and sticky session cannot be enabled. | v1.9 or later |
The following shows how to use the preceding annotations:
- Associate an existing load balancer. For details, see Using kubectl to Create a Service (Using an Existing Load Balancer).apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: performance # Load balancer typekubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithmspec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
- Automatically create a load balancer. For details, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).
Shared load balancer:
apiVersion: v1kind: Servicemetadata:annotations:kubernetes.io/elb.class: unionkubernetes.io/elb.autocreate: '{"type": "public","bandwidth_name": "cce-bandwidth-1551163379627","bandwidth_chargemode": "bandwidth","bandwidth_size": 5,"bandwidth_sharetype": "PER","eip_type": "5_bgp"}'kubernetes.io/elb.enterpriseID: '0' # ID of the enterprise project to which the load balancer belongskubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithmlabels:app: nginxname: nginxspec:ports:- name: service0port: 80protocol: TCPtargetPort: 80selector:app: nginxtype: LoadBalancerDedicated load balancer:
apiVersion: v1kind: Servicemetadata:name: nginxlabels:app: nginxnamespace: defaultannotations:kubernetes.io/elb.class: performancekubernetes.io/elb.autocreate: '{"type": "public","bandwidth_name": "cce-bandwidth-1626694478577","bandwidth_chargemode": "bandwidth","bandwidth_size": 5,"bandwidth_sharetype": "PER","eip_type": "5_bgp","available_zone": [""],"l4_flavor_name": "L4_flavor.elb.s1.small"}'kubernetes.io/elb.enterpriseID: '0' # ID of the enterprise project to which the load balancer belongskubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithmspec:selector:app: nginxports:- name: cce-service-0targetPort: 80nodePort: 0port: 80protocol: TCPtype: LoadBalancer
Sticky Session
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.session-affinity-mode | String | Source IP address-based sticky session means that access requests from the same IP address are forwarded to the same backend server.
NOTE: When kubernetes.io/elb.lb-algorithm is set to SOURCE_IP (source IP hash), sticky session cannot be enabled. | v1.9 or later |
kubernetes.io/elb.session-affinity-option | Sticky session timeout. | v1.9 or later |
The following shows how to use the preceding annotations:
apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: union # Load balancer typekubernetes.io/elb.session-affinity-mode: SOURCE_IP # The sticky session type is source IP address.kubernetes.io/elb.session-affinity-option: '{"persistence_timeout": "30"}' # Stickiness duration, which is measured in minutesspec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
Health Check
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.health-check-flag | String | Whether to enable the ELB health check.
If this parameter is enabled, the kubernetes.io/elb.health-check-option field must also be specified. | v1.9 or later |
kubernetes.io/elb.health-check-option | ELB health check configuration items. | v1.9 or later | |
kubernetes.io/elb.health-check-options | ELB health check configuration items. Each Service port can be configured separately, and you can configure only some ports. NOTE: Either kubernetes.io/elb.health-check-option or kubernetes.io/elb.health-check-options can be configured. | v1.19.16-r5 or later v1.21.8-r0 or later v1.23.6-r0 or later v1.25.2-r0 or later |
- The following shows how to use kubernetes.io/elb.health-check-option:apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: union # Load balancer typekubernetes.io/elb.health-check-flag: 'on' # Enable ELB health check.kubernetes.io/elb.health-check-option: '{"protocol":"TCP","delay":"5","timeout":"10","max_retries":"3"}'spec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
- For details about how to use kubernetes.io/elb.health-check-options, see Configuring Health Check on Multiple LoadBalancer Service Ports.
HTTP or HTTPS
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.protocol-port | String | If a Service is HTTP/HTTPS-compliant, configure the protocol and port number in the format of "protocol:port". where,
| v1.19.16 or later |
kubernetes.io/elb.cert-id | String | ID of an ELB certificate, which is used as the HTTPS server certificate. To obtain the certificate, log in to the CCE console, choose Service List > Networking > Elastic Load Balance, and click Certificates in the navigation pane. In the load balancer list, copy the ID under the target certificate name. | v1.19.16 or later |
For details about application scenarios and use cases, see Configuring HTTP/HTTPS for a LoadBalancer Service.
SNI
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.tls-certificate-ids | String | In ELB, the IDs of SNI certificates that must contain a domain name are separated by commas (,). To obtain the certificate, log in to the CCE console, choose Service List > Networking > Elastic Load Balance, and click Certificates in the navigation pane. In the load balancer list, copy the ID under the target certificate name. | v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
HTTPS must be enabled. For details, see Configuring SNI for a LoadBalancer Service.
Dynamic Adjustment of the Weight of the Backend ECS
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.adaptive-weight | String | Dynamically adjust the weight of the load balancer backend server based on the number pods on the server. In this way, the requests received by each pod are more balanced.
| v1.21 or later |
This parameter is invalid in passthrough networking, where dedicated load balancers are used in a CCE Turbo cluster.
The following shows how to use the preceding annotations:
apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: union # Load balancer typekubernetes.io/elb.adaptive-weight: 'true' # Enable dynamic adjustment of the weight of the backend ECS.spec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
Passthrough Capability
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.pass-through | String | Whether the access requests from within the cluster to the Service pass through the ELB load balancer. | v1.19 or later |
For details about application scenarios and use cases, see Configuring Passthrough Networking for a LoadBalancer Service.
Blocklist/Trustlist
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.acl-id | String |
| v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
kubernetes.io/elb.acl-status | String | This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:
| v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
kubernetes.io/elb.acl-type | String | This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:
| v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
The following shows how to use the preceding annotations:
apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: performance # Load balancer typekubernetes.io/elb.acl-id: <your_acl_id> # ID of an IP address group for accessing a load balancerkubernetes.io/elb.acl-status: 'on' # Enable access control.kubernetes.io/elb.acl-type: 'white' # Trustlist for access controlspec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
For details about application scenarios and use cases, see Configuring a Blocklist/Trustlist Access Policy for a LoadBalancer Service.
Host Network
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/hws-hostNetwork | String | If the pod uses hostNetwork, the ELB forwards the request to the host network after this annotation is used. Options:
| v1.9 or later |
The following shows how to use the preceding annotations:
apiVersion: v1kind: Servicemetadata:name: nginxannotations:kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value.kubernetes.io/elb.class: union # Load balancer typekubernetes.io/hws-hostNetwork: 'true' # The load balancer forwards the request to the host network.spec:selector:app: nginxports:- name: service0port: 80protocol: TCPtargetPort: 80type: LoadBalancer
Timeout
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.keepalive_timeout | String | Timeout for client connections. If there are no requests reaching the load balancer during the timeout duration, the load balancer will disconnect the connection from the client and establish a new connection when there is a new request. Value:
| Dedicated load balancers: v1.19.16-r30, v1.21.10-r10, v1.23.8-r10, v1.25.3-r10, or later Shared load balancers: v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
kubernetes.io/elb.client_timeout | String | Timeout for waiting for a request from a client. There are two cases:
The value ranges from 1 to 300 (in seconds). The default value is 60. | v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
kubernetes.io/elb.member_timeout | String | Timeout for waiting for a response from a backend server. After a request is forwarded to the backend server, if the backend server does not respond within the duration specified by member_timeout, the load balancer will stop waiting and return HTTP 504 Gateway Timeout. The value ranges from 1 to 300 (in seconds). The default value is 60. | v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
For details about application scenarios and use cases, see Configuring Timeout for a LoadBalancer Service.
Resource Tags
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.tags | String | Add resource tags to a load balancer. This parameter can be configured only when a load balancer is automatically created. A tag is in the format of "key=value". Use commas (,) to separate multiple tags. | v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later |
For details about application scenarios and use cases, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).
HTTP/2
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.http2-enable | String | Whether HTTP/2 is enabled. Request forwarding using HTTP/2 improves the access performance between your application and the load balancer. However, the load balancer still uses HTTP/1.x to forward requests to the backend server. Options:
Note: HTTP/2 can be enabled or disabled only when the listener uses HTTPS. This parameter is invalid and defaults to false when the listener protocol is HTTP. | v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
For details about application scenarios and use cases, see Configuring HTTP/2 for a LoadBalancer Service.
Obtaining Client IP Addresses
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.transparent-client-ip | String | This parameter is available only when a shared load balancer is used to create a LoadBalancer Service that is TCP/UDP-compliant.
| v1.23.17-r0, v1.25.12-r0, v1.27.9-r0, v1.28.7-r0, v1.29.3-r0, or later |
For details about application scenarios and use cases, see Enabling a LoadBalancer Service to Obtain the Client IP Address.
Changing a Custom EIP
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.custom-eip-id | String | ID of the custom EIP, which can be seen on the EIP console The EIP must be bindable. | v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later |
For details about application scenarios and use cases, see Changing a Custom EIP for a LoadBalancer Service.
Configuring Port Ranges for a Listener
Parameter | Type | Description | Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.port-ranges | String | If a dedicated load balancer is used and the TCP, UDP, or TLS protocol is selected, you can create a listener that listens to ports within a certain range from 1 to 65535. You can add a maximum of 10 port ranges that do not overlap for each listener. The parameter value is in the following format, where ports_name and port must be unique:
For example, the port names are cce-service-0 and cce-service-1, and the listener listens to ports 100–200 and 300–400, and 500–600 and 700–800, respectively.
NOTE: This function relies on ELB capabilities. Before using this function, check whether it is supported in the current region. For details about the regions where this function is supported, see . | v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later |
For details about application scenarios and use cases, see Configuring a Range of Listening Ports for LoadBalancer Services.
Parameters for Automatically Creating a Load Balancer
Parameter | Mandatory | Type | Description |
---|---|---|---|
name | No | String | Name of the automatically created load balancer. The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed. Default: cce-lb+service.UID |
type | No | String | Network type of the load balancer.
Default: inner |
bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is cce-bandwidth-******. The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed. |
bandwidth_chargemode | No | String | Bandwidth mode.
Default: bandwidth |
bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. Configure this parameter based on the bandwidth range allowed in your region. The minimum increment for bandwidth adjustment varies depending on the bandwidth range.
|
bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth sharing mode.
|
eip_type | Yes for public network load balancers | String | EIP type.
The specific type varies with regions. For details, see the EIP console. |
vip_subnet_cidr_id | No | String | The ID of the IPv4 subnet where the load balancer resides. This subnet is used to allocate IP addresses for the load balancer to provide external services. The IPv4 subnet must belong to the cluster's VPC. If this parameter is not specified, the load balancer and the cluster will be in the same subnet by default. This field can be specified only for clusters of v1.21 or later. How to Obtain Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the IPv4 Subnet ID on the Summary tab page. |
ipv6_vip_virsubnet_id | No | String | The ID of the IPv6 subnet where the load balancer is deployed. IPv6 must be enabled for the subnet. This parameter is available only for dedicated load balancers. How to Obtain Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the Network ID on the Summary tab page. |
elb_virsubnet_ids | No | Array of strings | The network ID of the subnet where the load balancer is located. This subnet is used to allocate IP addresses for accessing the backend server. If this parameter is not specified, the subnet specified by vip_subnet_cidr_id will be used by default. Load balancers occupy varying numbers of subnet IP addresses based on their specifications. Do not use the subnet CIDR blocks of other resources (such as clusters or nodes) as the load balancer's CIDR block. This parameter is available only for dedicated load balancers. Example:
How to Obtain Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the Network ID on the Summary tab page. |
vip_address | No | String | Private IP address of the load balancer. Only IPv4 addresses are supported. The IP address must be in the ELB CIDR block. If this parameter is not specified, an IP address will be automatically assigned from the ELB CIDR block. This parameter is available only in clusters of v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later versions. |
available_zone | Yes | Array of strings | AZ where the load balancer is located. This parameter is available only for dedicated load balancers. |
l4_flavor_name | No | String | Flavor name of the layer-4 load balancer. This parameter is mandatory when TCP or UDP is used.
This parameter is available only for dedicated load balancers. |
l7_flavor_name | No | String | Flavor name of the layer-7 load balancer. This parameter is mandatory when HTTP is used.
This parameter is available only for dedicated load balancers. Its value must match that of l4_flavor_name, meaning both must be either elastic specifications or fixed specifications. |
Parameter | Mandatory | Type | Description |
---|---|---|---|
delay | No | String | Health check interval (s) Value range: 1 to 50. Default value: 5 |
timeout | No | String | Health check timeout, in seconds. Value range: 1 to 50. Default value: 10 |
max_retries | No | String | Maximum number of health check retries. Value range: 1 to 10. Default value: 3 |
protocol | No | String | Health check protocol. Options: TCP, UDP, or HTTP |
path | No | String | Health check URL. This parameter needs to be configured when the protocol is HTTP. Default value: / Value range: 1-80 characters |
Parameter | Mandatory | Type | Description |
---|---|---|---|
target_service_port | Yes | String | Port for health check specified by spec.ports. The value consists of the protocol and port number, for example, TCP:80. |
monitor_port | No | String | Re-specified port for health check. If this parameter is not specified, the service port is used by default. NOTE: Ensure that the port is in the listening state on the node where the pod is located. Otherwise, the health check result will be affected. |
delay | No | String | Health check interval (s) Value range: 1 to 50. Default value: 5 |
timeout | No | String | Health check timeout, in seconds. Value range: 1 to 50. Default value: 10 |
max_retries | No | String | Maximum number of health check retries. Value range: 1 to 10. Default value: 3 |
protocol | No | String | Health check protocol. Default value: protocol of the associated Service Options: TCP, UDP, and HTTP |
path | No | String | Health check URL. This parameter needs to be configured when the protocol is HTTP. Default value: / Value range: 1-80 characters |
Parameter | Mandatory | Type | Description |
---|---|---|---|
persistence_timeout | Yes | String | Sticky session timeout, in minutes. This parameter is valid only when elb.session-affinity-mode is set to SOURCE_IP. Value range: 1 to 60. Default value: 60 |
- Interconnection with ELB
- Sticky Session
- Health Check
- HTTP or HTTPS
- SNI
- Dynamic Adjustment of the Weight of the Backend ECS
- Passthrough Capability
- Blocklist/Trustlist
- Host Network
- Timeout
- Resource Tags
- HTTP/2
- Obtaining Client IP Addresses
- Changing a Custom EIP
- Configuring Port Ranges for a Listener
- Parameters for Automatically Creating a Load Balancer