Managing JMeter Test Reports

Test Report Description

• Statistical dimension: In this report, RPS, response time, and concurrency are measured in a single thread. If a request has multiple packets, the request is considered successful only when all the packets are responded to. The response time of the request is the sum of the response time of the packets.
• Response timeout: If the corresponding TCP connection does not return the response data within the set response timeout (customized in *.jmx files), the thread request is counted as a response timeout. Possible causes include: the tested server is busy or crashes, or the network bandwidth is fully occupied.
• Verification failure: By default, the expected response code of HTTP/HTTPS is 200. However, when the response packet content returned from the server does not meet the expectation, response code such as 404 or 502 is returned. A possible cause is that the tested service cannot be processed normally in scenarios with a large number of concurrent users. For example, a database bottleneck exists in the distributed system or the backend application returns an error.
• Bandwidth: This report collects statistics on the bandwidth of the execution end of the performance test service. The uplink indicates the traffic sent from the performance test service, and the downlink indicates the received traffic. In the external pressure test scenario, you need to check whether the EIP bandwidth of the executor meets the uplink bandwidth requirement and whether the bandwidth exceeds the downlink bandwidth of 1 GB.
• RPS: RPS is the number of requests that CPTS sends to the tested server per second.
• How to determine the quality of tested applications: According to the service quality definition of an application, the optimal status is that there is no response or verification failure. If there is any response or verification failure, it must be within the defined service quality range. Generally, the value does not exceed 1%. The shorter the response time is, the better. User experience is considered good if the response time is within 2s, acceptable if it is within 5s and needs optimization when it is over 5s. TP90 and TP99 can objectively reflect the response time experienced by 90% to 99% of users.

Viewing a Real-Time Test Report

1. Log in to the CPTS console. In the navigation pane on the left, choose JMeter Test Projects. Click Edit Test Plan next to the project or click More > View Real-Time Report.
2. On the Real-Time Reports tab page, select the test plan whose test report you want to view.

   Click Stop Task to stop the test plan of a real-time report.

3. Select the thread under the test plan and view the report. For details about the parameters, see Table 1.
   Note

   • By default, the report of the first thread is displayed. If no thread is selected from the drop-down list, the reports of all threads are displayed.
   • You can select multiple threads under the test plan to view the reports of these threads.
   • You can click the drop-down list box before the thread name under Test metrics, and select one or more packets of the thread to view performance metrics of these packets.

4. View the response distribution statistics, bandwidth, response time ratio, TP (ms), and number of concurrent users.
5. View the monitoring metrics of the test plan, and perform calling analysis on the test plan. For details about the parameters, see Table 1.
   • Monitoring metrics: CPU, memory, disk read rate, and disk write rate.
   • Tracing analysis: supports analysis of the failed tracing and topology relationship to locate problems during the test.

Viewing an Offline Test Report

1. Log in to the CPTS console. In the navigation pane on the left, choose JMeter Test Projects. Click Edit Test Plan next to the project or click More > View Offline Report.
2. On the Offline Reports tab page, select the test plan whose report you want to view.
3. (Optional) Click in front of the report name to view the test plan definition and thread definition.
4. Click View Report, select a thread from the drop-down list, and view the test report result. For details about the parameters, see Table 1.
   Note

   • By default, the report of the first thread is displayed. If no thread is selected from the drop-down list, the reports of all threads are displayed.
   • You can select multiple threads under the test plan to view the reports of these threads.
   • You can click the drop-down list box before the thread name under Test metrics, and select one or more packets of the thread to view performance metrics of these packets.

5. View the response distribution statistics, bandwidth, response time ratio, TP (ms), and number of concurrent users.
   Note

   Monitoring data in offline reports can be stored for 30 days, but failed tracings and topology relationships can only be stored for 7 days.