Облачная платформаAdvanced

DLI Delta Table Overview

Delta tables use the Delta Lake technology to offer a robust data storage solution. By extending Parquet data files with file-based transaction logs, they support ACID transactions and scalable metadata. Delta Lake seamlessly integrates with Apache Spark APIs and is designed to work closely with structured streaming, allowing for the use of a single data backup for both batch and streaming operations, and enabling large-scale incremental processing.

Constraints on Delta in DLI

  • Only Spark 3.3.1 (3.0.0) or later supports Delta.
  • Only Delta 2.3.0 is supported by DLI.
  • Certain SQL statements in Spark 3.3.1 (3.0.0) do not support the open-source syntax related to Delta tables. For details, see Table 1.
    Table 1 Open source syntax related to Delta tables not supported by Spark 3.3.1-3.0.0 SQL

    Unsupported Statement

    Example

    ALTER TABLE REPLACE COLUMNS: replaces columns

    alter table table0 replace columns(id1 int,name1 string);

    SHOW CREATE TABLE: shows table creation statements

    show create table table1;

    INSERT INTO/OVERWRITE: inserts data into a table in a specified static partition

    insert into table1 partition(part='part1') select * from table2;

    ALTER TABLE ADD/DROP PARTITION: manages partitions

    alter table test_delta_parts1 add partition('2024-10-28');

    CONVERT TO DELTA: does not support tables in parquet.'tablePath' format

    convert to delta parquet.`obs://bucket0/db0/table0`;
Следующая статья
Using Delta to Develop Jobs in DLI
Поддержка Юридические документы Политика конфиденциальности
© 2026 Cloud.ru