How Do I Enable NIC Multi-Queue for an Image?

Scenarios

ECSs Supporting NIC Multi-Queue

• For details about the ECS flavors that support NIC multi-queue, see section "Instances" in Elastic Cloud Server User Guide.
  Note

  If there are more than 1 NIC queue, NIC multi-queue is supported.

• Only KVM ECSs support NIC multi-queue.
• The Linux public images listed in Table 2 support NIC multi-queue.
  Note

  • Windows has not commercially supported NIC multi-queue. If you enable NIC multi-queue for a Windows image, an ECS created from such an image may take longer than normal to start.
  • For Linux ECSs, you are advised to upgrade the kernel to 2.6.35 or later. Otherwise, NIC multi-queue is not supported.

    Run uname -r to check the kernel version. If the version is earlier than 2.6.35, contact technical support to upgrade it.

Operation Instructions

• If the ECS was created from a public image listed in ECSs Supporting NIC Multi-Queue, NIC multi-queue is enabled on the ECS by default. You do not need to enable NIC multi-queue manually.
• If the ECS was created from an external image file with an OS listed in ECSs Supporting NIC Multi-Queue, you may need to perform the following operations to enable NIC multi-queue:
  1. Register the External Image File as a Private Image.
  2. Enable NIC Multi-Queue for the Image.
  3. Create an ECS from the Private Image.
  4. Enable NIC Multi-Queue on the ECS.

Register the External Image File as a Private Image

Enable NIC Multi-Queue for the Image

Method 1:

1. Log in to the management console.
   1. Under Computing, click Image Management Service. The IMS console is displayed.
2. On the displayed Private Images page, locate the row that contains the image and click Modify in the Operation column.
3. Enable NIC multi-queue for the image.

Method 2:

1. Log in to the management console.
   1. Under Computing, click Image Management Service. The IMS console is displayed.
2. On the displayed Private Images page, click the name of the image.
3. In the upper right corner of the displayed image details page, click Modify. In the displayed Modify Image dialog box, enable NIC multi-queue for the image.

1. Obtain a token. For details, see Calling APIs > Authentication in Image Management Service API Reference.
2. Call an API to update an image. For details, see "Modifying an Image (Native OpenStack API)" in Image Management Service API Reference.
3. Add X-Auth-Token to the request header.

   The value of X-Auth-Token is the token obtained in step 1.

4. Add Content-Type to the request header.

   Set Content-Type to application/openstack-images-v2.1-json-patch.

   The request URI is in the following format:

   PATCH /v2/images/{image_id}

   The request body is as follows:

   [       
            { 
             "op":"add",
             "path":"/hw_vif_multiqueue_enabled", 
             "value": true
            } 
    ]
   
   

Create an ECS from the Private Image

Use the registered private image to create an ECS. For details, see the Elastic Cloud Server User Guide. Note the following when setting the parameters:

• Region: Select the region where the private image is located.
• Image: Select Private image and then the desired image from the drop-down list.

Enable NIC Multi-Queue on the ECS

1. Enable NIC multi-queue.
   1. Log in to the ECS.
   2. Run the following command to obtain the number of queues supported by the NIC and the number of queues with NIC multi-queue enabled:

      ethtool -l NIC

   3. Run the following command to configure the number of queues used by the NIC:

      ethtool -L NIC combined Number of queues

   Example:

   [root@localhost ~]# ethtool -l eth0  #View the number of queues used by NIC eth0eth0.
   Channel parameters for eth0:
   Pre-set maximums:
   RX:               0
   TX:               0
   Other:                  0
   Combined: 4  #Indicates that a maximum of four queues can be enabled for the NIC.
   Current hardware settings:
   RX:               0
   TX:               0
   Other:                  0
   Combined: 1 #Indicates that one queue has been enabled.
   
   [root@localhost ~]# ethtool -L eth0 combined 4 #Enable four queues on NIC eth0eth0.
   
   

2. (Optional) Enable irqbalance so that the system automatically allocates NIC interrupts to multiple vCPUs.
   1. Run the following command to enable irqbalance:

      service irqbalance start

   2. Run the following command to view the irqbalance status:

      service irqbalance status

      If the Active value in the command output contains active (running), irqbalance has been enabled.

      Figure 1 Enabled irqbalance

      
      

3. (Optional) Enable interrupt binding.

   Enabling irqbalance allows the system to automatically allocate NIC interrupts, improving network performance. If the improved network performance fails to meet your expectations, manually configure interrupt affinity on the target ECS.

   The detailed operations are as follows:

   Run the following script so that each ECS vCPU responds the interrupt requests initialized by one queue. That is, one queue corresponds to one interrupt, and one interrupt binds to one vCPU.

   #!/bin/bash
   service irqbalance stop
   
   eth_dirs=$(ls -d /sys/class/net/eth*)
   if [ $? -ne 0 ];then
       echo "Failed to find eth*  , sleep 30" >> $ecs_network_log
       sleep 30
       eth_dirs=$(ls -d /sys/class/net/eth*)
   fi
   
   for eth in $eth_dirs
   do
       cur_eth=$(basename $eth)
       cpu_count=`cat /proc/cpuinfo| grep "processor"| wc -l`
       virtio_name=$(ls -l /sys/class/net/"$cur_eth"/device/driver/ | grep pci |awk {'print $9'})
   
       affinity_cpu=0
       virtio_input="$virtio_name""-input"
       irqs_in=$(grep "$virtio_input" /proc/interrupts | awk -F ":" '{print $1}')
       for irq in ${irqs_in[*]}
       do
           echo $((affinity_cpu%cpu_count)) > /proc/irq/"$irq"/smp_affinity_list
           affinity_cpu=$[affinity_cpu+2]
       done
   
       affinity_cpu=1
       virtio_output="$virtio_name""-output"
       irqs_out=$(grep "$virtio_output" /proc/interrupts | awk -F ":" '{print $1}')
       for irq in ${irqs_out[*]}
       do
           echo $((affinity_cpu%cpu_count)) > /proc/irq/"$irq"/smp_affinity_list
           affinity_cpu=$[affinity_cpu+2]
       done
   done
   
   

4. (Optional) Enable XPS and RPS.

   XPS allows the system with NIC multi-queue enabled to select a queue by vCPU when sending a data packet.

   #!/bin/bash
   # enable XPS feature
   cpu_count=$(grep -c processor /proc/cpuinfo)
   dec2hex(){
     echo $(printf "%x" $1)
   }
   eth_dirs=$(ls -d /sys/class/net/eth*)
   if [ $? -ne 0 ];then
       echo "Failed to find eth* , sleep 30" >> $ecs_network_log
       sleep 30
       eth_dirs=$(ls -d /sys/class/net/eth*)
   fi
   for eth in $eth_dirs
   do
       cpu_id=1
       cur_eth=$(basename $eth)
       cur_q_num=$(ethtool -l $cur_eth | grep -iA5 current | grep -i combined | awk {'print $2'})
       for((i=0;i<cur_q_num;i++))
       do
           if [ $i -eq $ cpu_count ];then
               cpu_id=1
           fi
           xps_file="/sys/class/net/${cur_eth}/queues/tx-$i/xps_cpus"
           rps_file="/sys/class/net/${cur_eth}/queues/rx-$i/rps_cpus"
           cpuset=$(dec2hex "$cpu_id")
           echo $cpuset > $xps_file
           echo $cpuset > $rps_file
           let cpu_id=cpu_id*2
       done
   done