Облачная платформаAdvanced

Service Overview

Эта статья полезна?

Язык статьи: Английский

In Kubernetes, a Service makes an application running on a set of pods network-accessible. It provides a consistent DNS name for these pods and distributes traffic across them for load balancing. This section describes the basic concepts of Kubernetes Services and provides a comparison of various Service types.

Service

After a pod is created, accessing it directly can result in certain problems:

The pod can be deleted and recreated at any time by a controller such as a Deployment. If the pod is recreated, access to it may fail.

An IP address cannot be assigned to a pod until the pod is started. Before the pod is started, its IP address is unknown.

Applications usually run on multiple pods that use the same image. Accessing pods one by one is not efficient.

For example, a Deployment is used to deploy the frontend and backend of an application. The frontend calls the backend for computing, as shown in Figure 1. Three pods are running in the backend, and they are independent and replaceable. When a backend pod is recreated, the new pod is assigned with a new IP address, but the frontend pod is unaware of this change.

Figure 1 Inter-pod access

To solve these problems, Kubernetes introduces Services, which provide stable network interfaces and persistent IP addresses for pods. You can set these IP addresses using Service CIDR blocks during cluster creation. The Service CIDR blocks are used to assign IP addresses to Services. A Service uses a label selector to identify its target pods and uses load balancing to evenly distribute traffic among them. This abstraction eliminates the complexity of directly accessing individual pods and enhances both the availability and efficiency of workloads.

For the example above, a Service is created for the backend pods and used for the frontend pod to access these backend pods. In this way, the frontend pod remains unaffected by any changes to the backend pods, as shown in Figure 2.

Figure 2 Accessing pods through a Service

Endpoint and EndpointSlice

In Kubernetes, both Endpoints and EndpointSlices are resource objects used to manage the network endpoints of service backend pods.

Endpoints are a traditional resource object for service discovery. An Endpoint is a collection of endpoints of all pods associated with a Service. An Endpoint is automatically created for each Service with a selector defined. It records a list of IP addresses and ports for all matching pods. After a pod is deleted or recreated, its IP address may change. The Endpoint updates the pod IP address and port accordingly to ensure that the Service continues to route traffic to the correct pods.

Key features of Endpoints include:

Each Endpoint object corresponds to a single Service.

Each Endpoint stores the IP addresses and ports of all backend pods associated with the Service.

Any change to a pod's status triggers an update to the corresponding Endpoint object.

By default, an Endpoint object can hold up to 1000 endpoints. If this limit is exceeded, additional endpoints are truncated and not included in the object.

EndpointSlices are an alternative to the traditional Endpoints API, introduced as an alpha feature in Kubernetes v1.16 and promoted to stable in v1.21. They address the performance limitations of Endpoints in large clusters by sharding network endpoints.

Each Service is associated with one or more EndpointSlices.

By default, each EndpointSlice can hold up to 100 endpoints. When this limit is exceeded, Kubernetes creates additional EndpointSlices to accommodate the remaining endpoints.

Endpoints are grouped by protocol, port, and Service name.

EndpointSlices support IPv4, IPv6, and fully qualified domain name (FQDN) address types.

The differences between Endpoint and EndpointSlice objects are listed in the table below.

Category	Endpoint	EndpointSlice	Difference
Architecture	All endpoints are stored in a single object.	Endpoints are stored in different shards.	An EndpointSlice distributes endpoints in multiple objects.
Maximum number of endpoints per object	1000	100	An EndpointSlice supports more endpoints through shards.
Update granularity	Full object update required on any change	Only affected shards updated	When a pod is changed, the EndpointSlice only updates the affected shards.
Network traffic	High (entire object transmitted on any change)	Low (only changed shards transmitted)	They have significant differences in large clusters.
API resource consumption	High	Low	EndpointSlices reduce etcd pressure.
Kubernetes version	Supported in all versions (Endpoints will be deprecated in v1.33 and replaced by EndpointSlices.)	Introduced in v1.16, stable since v1.21	Clusters of earlier versions may not support EndpointSlices.
Application	Small clusters	Large clusters	EndpointSlices are designed for high-performance environments.

Request Forwarding (iptables, nftables, and IPVS)

kube-proxy is a key component of a Kubernetes cluster. It is used for load balancing and forwarding data between a Service and its backend pods. CCE supports the following Service forwarding modes: iptables, nftables (v1.35 and later), and IPVS.

nftables is a high-performance packet filtering and management framework built into the Linux kernel, designed to replace iptables. It eliminates performance bottlenecks and scalability limitations while providing more efficient and flexible network traffic control. Built on the Netfilter hook mechanism, nftables uses a unified configuration framework to integrate rule management across multiple protocols (IPv4, IPv6, ARP, and others), eliminating the need for separate per-protocol rule sets. It also optimizes rule matching through hash-based data structures, improving efficiency in high-concurrency scenarios.

In nftables mode, kube-proxy uses nftables instead of iptables to configure packet forwarding and NAT rules. Compared with iptables, nftables offers simpler syntax and more flexible rule organization. It supports incremental rule updates by modifying individual rules without replacing the entire rule set, thereby optimizing rule management performance in large-scale service scenarios and better accommodating Kubernetes cluster growth. For details, see nftables proxy mode.

iptables is a Linux kernel function for processing and filtering a large number of data packets. It allows flexible sequences of rules to be attached to various hooks in the packet processing pipeline. When iptables is used, kube-proxy implements NAT and load balancing in the NAT pre-routing hook. For each Service, kube-proxy installs an iptables rule which captures the traffic destined for the Service's ClusterIP and ports and redirects the traffic to one of the backend pods. By default, iptables randomly selects a backend pod. For details, see iptables proxy mode.

IPVS is constructed on top of Netfilter and balances transport-layer loads as part of the Linux kernel. IPVS can direct requests for TCP- or UDP-based services to the real servers, and make services of the real servers appear as virtual services on a single IP address.

In the IPVS mode, kube-proxy uses IPVS load balancing instead of iptables. IPVS is designed to balance loads for a large number of Services. It has a set of optimized APIs and uses optimized search algorithms instead of simply searching for rules from a list. For details, see IPVS proxy mode.

For details about the comparison between iptables, nftables, and IPVS, see Comparing iptables, nftables, and IPVS.

Service Affinity (externalTrafficPolicy)

For a NodePort and LoadBalancer Service, requests are first sent to the node port, then the Service, and finally the pod backing the Service. The backing pod may not be located in the node receiving the requests. By default, the backend workload can be accessed from any node IP address and service port. If the pod is not on the node that receives the request, the request will be redirected to the node where the pod is located, which may cause performance loss.

You can configure service affinity using the console or YAML.

Using the console: When creating the NodePort and LoadBalancer Services on the CCE console, you can configure service affinity using the Service Affinity option.

Using YAML: The externalTrafficPolicy parameter in a Service is used to determine whether the external traffic can be routed to the local nodes or cluster-wide endpoints. The following is an example:

apiVersion: v1
kind: Service
metadata:
  name: nginx-nodeport
spec:
  externalTrafficPolicy: Local # Service affinity
  ports:
  - name: service
    nodePort: 30000
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: NodePort

If the value of externalTrafficPolicy is Local, requests sent from <node-IP-address:Service-port> will be forwarded only to the pod on the local node. If the node does not have a pod, the requests are suspended.

If the value of externalTrafficPolicy is Cluster, requests are forwarded within the cluster and the backend workload can be accessed from any node IP address and service port.

If externalTrafficPolicy is not set, the default value Cluster will be used.

The table below compares the two options of service affinity (externalTrafficPolicy).

Table 1 Comparison of the two types of service affinity
Category	Cluster-level (Cluster)	Node-level (Local)
Application	This mode applies to scenarios where high performance is not required and the source IP address of the client does not need to be retained. This mode brings more balanced load to each node in the cluster.	This mode applies to scenarios where high performance is required and the source IP address of the client needs to be retained. However, traffic is forwarded only to the node where the container resides, and source IP address translation is not performed.
Access mode	The IP addresses and access ports of all nodes in a cluster can access the workload associated with the Service.	Only the IP address and access port of the node where the workload is located can access the workload associated with the Service.
Obtaining client source IP address	The source IP address of the client cannot be obtained.	The source IP address of the client can be obtained.
Access performance	Service access will cause performance loss due to route redirection, and the next hop for a data packet may be another node.	Service access will not cause performance loss due to route redirection.
Load balancing	Traffic propagation has good overall load balancing.	There is a potential risk of unbalanced traffic propagation.
Other special case	None	In different container network models and service forwarding modes, accessing Services from within the cluster may fail. For details, see Why a Service Fails to Be Accessed from Within the Cluster.

Service Types

You can create a specified type of Service in a cluster. The description and application scenarios of different types of Services are listed in the table below.

Service Type	Description	Application
ClusterIP	ClusterIP Services are the default type of Kubernetes Services. They assign a virtual IP address, accessible only within the cluster, from the cluster's Service CIDR block.	Services that only need to communicate with each other within a cluster and do not need to be exposed outside the cluster. For example, if a frontend application pod in a cluster needs to access a backend database in the same cluster, you can create a ClusterIP Service for the backend database to ensure that the backend database can be accessed only within the cluster.
NodePort	A NodePort Service opens a port on each node in a cluster to allow external traffic to access the Service through <node-IP-address>:<node-port>.	Scenarios where temporary or low-volume access is required. For example, in a testing environment, you can use a NodePort Service when deploying and debugging a web application.
LoadBalancer	A LoadBalancer Service adds an external load balancer on the top of a NodePort Service and distributes external traffic to multiple pods within a cluster. It automatically assigns an external IP address to allow access from the clients. LoadBalancer Services process TCP and UDP traffic at Layer 4 (transport layer) of the OSI model. They can be extended to support Layer 7 (application layer) capabilities to manage HTTP and HTTPS traffic.	Cloud applications that require a stable, easy-to-manage entry for external access. For example, in a production environment, you can use LoadBalancer Services to expose public-facing services such as web applications and API services to the Internet. These services often need to handle heavy traffic while maintaining high availability.
DNAT	A DNAT Service provides Network Address Translation (NAT) for all nodes in a cluster so that multiple nodes can share an EIP.	Services that require temporary or low-volume access from the Internet. DNAT Services provide higher reliability than NodePort Services. With a DNAT Service, there is no need to bind an EIP to a single node, and requests can still be distributed to the workload even if any of the nodes inside is down.
Headless Services	For headless Services, no cluster IP address is allocated. When a client attempts to access a backend pod and performs a DNS query for the Service, it receives a list of the IP addresses of the backend pods. This allows the client to communicate directly with individual pods.	Applications that require direct communication with specific backend pods instead of using proxies or load balancers. For example, when you deploy a stateful application (such as a ClickHouse database), you can use a headless Service. It allows application pods to directly access each ClickHouse pod and enables targeted read and write operations. This enhances overall data processing efficiency.

Why a Service Fails to Be Accessed from Within the Cluster

If the service affinity of a Service is set to the node level, that is, the value of externalTrafficPolicy is Local, the Service may fail to be accessed from within the cluster (specifically, nodes or containers). Information similar to the following is displayed:

upstream connect error or disconnect/reset before headers. reset reason: connection failure
Or
curl: (7) Failed to connect to **.**.**.** port 900: Connection refused
Or
curl: (28) Failed to connect to **.**.**.** port 1122: Connection timed out

It is common that a load balancer in a cluster cannot be accessed. The reason is as follows: When Kubernetes creates a Service, kube-proxy adds the access address of the load balancer as an external IP address (External-IP, as shown in the following command output) to iptables or IPVS. If a client inside the cluster initiates a request to access the load balancer, the address is considered as the external IP address of the Service, and the request is directly forwarded by kube-proxy without passing through the load balancer outside the cluster.

# kubectl get svc nginx
NAME    TYPE           CLUSTER-IP       EXTERNAL-IP                    PORT(S)        AGE
nginx   LoadBalancer   10.247.76.156    123.**.**.**,192.168.0.133    80:32146/TCP   37s

When the value of externalTrafficPolicy is Local, the access failures in different container network models and service forwarding modes are as follows:

Note

For a multi-pod workload, ensure that all pods are accessible. Otherwise, there is a possibility that the access to the workload fails.

The table lists only the scenarios where the access may fail. Other scenarios that are not listed in the table indicate that the access is normal.

For CCE clusters using IPVS, set externalTrafficPolicy to Local for node-level Service affinity. For CCE Turbo clusters with DataPlane V2 disabled, node-level affinity is supported only when the Service backend uses hostNetwork.

Server Service Type	Access Type	Client Request Source	Tunnel Network Cluster	VPC Network Cluster (DataPlane V2 disabled. If enabled, see Clusters with DataPlane V2 Enabled.)	CCE Turbo Cluster (DataPlane V2 disabled. If enabled, see Clusters with DataPlane V2 Enabled.)
NodePort Service	Public/Private network	Same node as service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Accessible
		Different node from service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on another node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node EIP + NodePort on the client node: Failure Access via node IP + NodePort on another node: Failure	Accessible
		Other containers on the same node as the service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Access via node EIP + NodePort on the server node: Success Access via node private EIP + NodePort on the server node: Failure Access via node IP + NodePort on a non-server node: Failure	Accessible
		Other containers on a different node from the service pod	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node IP + NodePort on another node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node IP + NodePort on another node: Failure	Accessible
LoadBalancer Service using a dedicated load balancer	Private network	Same node as service pod	Inaccessible	Inaccessible	Accessible
LoadBalancer Service using a dedicated load balancer	Private network	Other containers on the same node as the service pod	Inaccessible	Inaccessible	Accessible

For CCE clusters using iptables, set externalTrafficPolicy to Local for node-level Service affinity. For CCE Turbo clusters with DataPlane V2 disabled, node-level affinity is supported only when the Service backend uses hostNetwork.

Server Service Type	Access Type	Client Request Source	Tunnel Network Cluster	VPC Network Cluster (DataPlane V2 disabled. If enabled, see Clusters with DataPlane V2 Enabled.)	CCE Turbo Cluster (DataPlane V2 disabled. If enabled, see Clusters with DataPlane V2 Enabled.)
NodePort Service	Public/Private network	Same node as service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Accessible
		Different node from service pod	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node EIP + NodePort on the client node: Failure Access via node IP + NodePort on another node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node EIP + NodePort on the client node: Failure Access via node IP + NodePort on another node: Failure	Accessible
		Other containers on the same node as the service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Access via node EIP + NodePort on the server node: Success Access via node private EIP + NodePort on the server node: Failure Access via node IP + NodePort on a non-server node: Failure	Accessible
		Other containers on a different node from the service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on a non-server node: Failure	Accessible
LoadBalancer Service using a dedicated load balancer	Private network	Same node as service pod	Inaccessible	Inaccessible	Accessible
LoadBalancer Service using a dedicated load balancer	Private network	Other containers on the same node as the service pod	Inaccessible	Inaccessible	Accessible

For clusters with DataPlane V2 enabled, set externalTrafficPolicy to Local for node-level Service affinity.

Note

DataPlane V2 is supported only in CCE VPC-network clusters and Turbo clusters. Once enabled, it replaces kube-proxy entirely, eliminating the need to configure a service forwarding mode.

Server Service Type	Access Type	Client Request Source	VPC Network Cluster (DataPlane V2 Enabled to Replace kube-proxy)	CCE Turbo Cluster (DataPlane V2 Enabled to Replace kube-proxy)
NodePort Service	Public/Private network	Same node as service pod	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on a non-server node: Success Access via node EIP + NodePort on a non-server node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on a non-server node: Success Access via node EIP + NodePort on a non-server node: Failure
		Different node from service pod	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on a non-server node: Success Access via node EIP + NodePort on a non-server node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on a non-server node: Success Access via node EIP + NodePort on a non-server node: Failure
		Other containers on the same node as the service pod	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on another node: Failure	Access via node IP + NodePort on the server node: Success Access via node IP + NodePort on another node: Failure
		Other containers on a different node from the service pod	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node IP + NodePort on another node: Failure	Access via node IP + NodePort on the server node: Success Access via node private IP + NodePort on the client node: Success Access via node IP + NodePort on another node: Failure
LoadBalancer Service using a dedicated load balancer	Private network	Same node as service pod	Inaccessible	Inaccessible
LoadBalancer Service using a dedicated load balancer	Private network	Other containers on the same node as the service pod	Inaccessible	Accessible

The following methods can be used to solve this problem:

(Recommended) In the cluster, use the ClusterIP Service or service domain name for access.

Set externalTrafficPolicy of the Service to Cluster, which means cluster-level service affinity. Note that this affects source address persistence.

Assume that the Service name is my-service and it is in the default namespace. The kubectl command example is as follows:

kubectl patch service my-service -n default --type='merge' -p '{"spec":{"externalTrafficPolicy":"Cluster"}}'

Leveraging the pass-through feature of the Service, kube-proxy is bypassed when the ELB address is used for access. The ELB load balancer is accessed first, and then the workload. For details, see Configuring Passthrough Networking for a LoadBalancer Service.

Note

In a CCE standard cluster, after passthrough networking is configured using a dedicated load balancer, the private IP address of the load balancer cannot be accessed from the node where a workload pod resides or other pods on the same node as the workload.

This function is only available in clusters v1.19 and later.

In IPVS, the passthrough settings of Services connected to the same load balancer must be the same.

If node-level (local) service affinity is used, kubernetes.io/elb.pass-through is automatically set to onlyLocal to enable passthrough networking.

Parent topic: Services

Эта статья полезна?

Поддержка Юридические документы Политика конфиденциальности