Creating a Non-Elastic Resource Pool Queue (Discarded and Not Recommended)
Queues in the non-elastic resource pool mode are the previous-gen of resource management for DLI. It involved purchasing and releasing resources based on usage demands, requiring estimation of resource needs before making purchases.
Queues in an elastic resource pool are recommended, as they offer the flexibility to use resources with high utilization as needed. For how to buy an elastic resource pool and create queues within it, see Creating an Elastic Resource Pool and Creating Queues Within It.
- If you use a sub-account to create a queue for the first time, log in to the DLI management console using the main account and keep records in the DLI database before creating a queue.
- It takes 6 to 10 minutes for a job running on a new queue for the first time.
- After a queue is created, if no job is run within one hour, the system releases the queue.
- Queues with 16 CUs do not support scale-out or scale-in.
- Queues with 64 CUs do not support scale-in.
Notes and Constraints
Item | Description |
---|---|
Resource type |
|
Managing queues |
|
Queue scaling |
|
Procedure
- You can create a queue on the Overview, SQL Editor, or Queue Management page.
- In the upper right corner of the Overview page, click Create Queue.
- To create a queue on the Queue Management page:
- In the navigation pane on the left of the DLI management console, choose Resources > Queue Management.
- In the upper right corner of the Queue Management page, click Create Queue to create a queue.
- To create a queue on the SQL Editor page:
- In the navigation pane of the DLI management console, click SQL Editor.
- Click Queues. On the tab page displayed, click
on the right to create a queue.
- On the Create Queue page displayed, set the parameters according to Table 2.
Table 2 Parameters Parameter
Description
Name
Queue name
- Only numbers, letters, and underscores (_) are allowed. The value cannot contain only numbers, start with an underscore (_), or be left unspecified.
- The value can contain a maximum of 128 characters.
NOTE:The queue name is case-insensitive. Uppercase letters will be automatically converted to lowercase letters.
Type
- For SQL: compute resources used for SQL jobs.
- For general purpose: compute resources used for Spark and Flink jobs.NOTE:
When a dedicated queue is not in use, its resources are still reserved and not released. This means that resources are constantly being held, regardless of whether the queue is being utilized. By using a dedicated queue, resources can be guaranteed to be available whenever jobs are submitted. You can create enhanced datasource connections for a dedicated queue.
Specifications
The compute nodes' total number of CUs. One CU equals one vCPU and 4 GB of memory. DLI automatically assigns CPU and memory resources to each compute node, and the client does not need to know how many compute nodes are being used.
Description
Description of the queue to be created. The value can contain a maximum of 128 characters.
Advanced Settings
In the Queue Type area, select Dedicated Resource Mode and then click Advanced Settings.
- Default: The system automatically configures the parameter.
- Custom
CIDR Block: Enter a CIDR block range. If DLI enhanced datasource connection is used, the CIDR block of the DLI queue cannot overlap with that of the data source.
Queue Type: When running an AI-related SQL job, select AI-enhanced. When running other jobs, select Basic.
- Click Create Now to create a queue.
Once created, you can check and use it on the Queue Management page.
NoteIt takes 6 to 10 minutes for a job running on a new queue for the first time.
- Notes and Constraints
- Procedure