Modifying Doris Parameters
Scenario
Modify the Doris configuration parameters of the cluster on the CloudTable console to optimize the cluster performance.
- Restart the cluster after static parameter modification. Otherwise, services will be interrupted.
- You can modify parameters only when no task is running in the cluster.
- Do not modify cluster parameters when the cluster is being restarted.
Procedure
- Log in to the CloudTable console.
- Select a region in the upper left corner.
- In the navigation pane on the left, click Cluster Management.
- In the cluster list, find the target cluster and click the cluster name. The cluster details page is displayed.
- In the Parameter Configuration area, click the Parameter Configuration tab and modify Doris parameters.
- Select the target FE or BE node, select the parameter to be modified, and click
in the Value column. - Enter a new value in the text box, and then click
. The parameter value has been successfully changed if the system prompts "The parameter changed to xx successfully. Save the modified value." The new parameter value is marked with a red asterisk (*).If you want to cancel it, click
.Figure 1 Modifying a parameter
- Click Save Changes in the upper left corner of the parameter list. The Save Changes dialog box is displayed.
- In the Save Changes dialog box, verify whether the parameter settings are correct and click OK.
- Click OK and click
in the upper right corner. You do not need to restart the cluster if the parameter turns into the effective state. Otherwise, manually restart the cluster for the modification to take effect. - Click the Restart button located in the upper-left corner of the Parameter Configuration tab to restart the cluster. Alternatively, you can navigate back to the cluster list page to restart the cluster.Note
Manually restart the cluster after the static parameters are modified. Dynamic parameters take effect after being modified. You do not need to restart the cluster.
Figure 2 Restarting the cluster
- If the new parameter value is incorrect, click Quick Restore to cancel the modification.
- Click OK and click
- After modifying parameters, you can click the Change History tab to view the change history. The following table lists the parameters displayed on the Change History page.
Table 1 Parameters Parameter
Description
Name
Name of the modified parameter
Old Value
Parameter value before modification
New Value
Parameter value after modification
Modified
Time when you modify a parameter value
Doris Parameter Modification Status
Status | Description |
|---|---|
Unmodified | The parameter has not been modified. |
Unapplied | The parameter has been modified but has not been applied. |
Applying | This is a static parameter and you need to restart the cluster after saving the modification. |
Applied | The modification of the parameter takes effect.
|
FE Node Parameters
Parameter | Value | Static | Type | Status | Value Range | Default Value | Description |
|---|---|---|---|---|---|---|---|
max_routine_load_job_num | 100 | No | Int | Unmodified | (0, 300) | 100 | Specifies the total number of routine import jobs, including the jobs in the NEED_SCHEDULED, RUNNING, and PAUSE states. After this parameter value is exceeded, no new jobs can be submitted. |
use_new_tablet_scheduler | true | Yes | Boolean | Unmodified | true or false | true | Whether to enable the new replica scheduling mode. |
max_broker_concurrency | 10 | No | Int | Unmodified | (5, 30) | 10 | Maximal concurrency of broker scanners. |
tablet_repair_delay_factor_second | 60 | No | Long | Unmodified | (0,9223372036854775807) | 60 | For different scheduling priorities, we will delay different time to start repairing in order to prevent a large number of unnecessary replica repair tasks from occurring in the process of routine restart and upgrade. This parameter is a reference coefficient. For HIGH priority, the delay is the reference coefficient multiplied by 1; for NORMAL priority, the delay is the reference coefficient multiplied by 2; for LOW priority, the delay is the reference coefficient multiplied by 3. That is, the lower the priority, the longer the delay waiting time. If the user wants to repair the replica as soon as possible, this parameter can be reduced appropriately. |
balance_load_score_threshold | 0.1 | No | Double | Unmodified | (0, 1) | 0.1 | Threshold of the cluster balance percentage. The default value is 0.1, that is, 10%. When the load core of a BE node is not higher than or less than 10% of the average load core, we think that the node is balanced. If you want to make the cluster load more even, you can adjust this parameter appropriately. |
dynamic_partition_check_interval_seconds | 600 | No | Long | Unmodified | (0,9223372036854775807 | 600 | Specifies the execution frequency of dynamic partition threads. The default value is 600 (10 minutes), that is, scheduling is performed every 10 minutes. Unit: second |
max_routine_load_task_concurrent_num | 5 | No | Int | Unmodified | (0, 10) | 5 | This parameter limits the maximum number of concurrent subtasks for a routine import job. It is advised to keep the default value. Setting it too large may result in too many concurrent tasks and consume cluster resources. |
dynamic_partition_enable | true | No | Boolean | Unmodified | true or false | true | Whether to enable Doris's dynamic partition feature. The default value is false, indicating that the dynamic partition feature is disabled. This parameter only affects the partitioning operation of dynamic partition tables, not normal tables. |
query_detail_history_storage_life_time_in_days | 7 | No | Int | Unmodified | (0,31) | 7 | Storage duration of slow query records. Records that exceed the storage duration will be deleted. Unit: day |
max_bytes_per_broker_scanner | 3221225472 | No | Long | Unmodified | (0,9223372036854775807) | 3221225472 | Maximum bytes a broker scanner can process in one broker load job. If the amount of data imported by a single BE is greater than the value of max_bytes_per_broker_scanner (3 GB by default), an error is reported. In this case, you need to increase the value of max_bytes per_broker_scanner or adjust the number of concurrent import tasks. The calculation logic is as follows: Number of concurrent import tasks = Math.min (Source file size/Minimum processing volume min_bytes_per_broker scanner, Maximum number of concurrent tasks max broker concurrency, Number of current BE nodes) Processing volume of a single BE = Source file size/Number of concurrent import tasks Unit: byte |
BE node parameters
Parameter | Value | Static | Type | Status | Value Range | Default Value | Description |
|---|---|---|---|---|---|---|---|
max_consumer_num_per_grou | 3 | Yes | Int | Unmodified | (0, 10) | 3 | Maximum number of consumers in a data consumer group, used for routine load. |
clone_worker_count | 3 | Yes | Int | Unmodified | (0, 10) | 3 | Number of threads used to perform cloning tasks. The value affects the speed of replica equalization. In the case of low disk pressure, you can speed up replica balancing by adjusting this parameter. |
- Scenario
- Procedure
- Doris Parameter Modification Status
- FE Node Parameters
- BE node parameters