nav-img
Advanced

When Using Spark 3.3.1 Client to Create a View and Perform a Join Query, an Error Occurs Stating "Not allowed to create a permanent view" When Attempting to Write the Data

Symptom

When using Spark 3.3.1 client to create a subquery view with aggregate functions, an error occurs when attempting to write the data, stating "Not allowed to create a permanent view without explicitly assigning an alias for expression count".

Cause Analysis

To address schema compatibility issues caused by inconsistent default aliases in different version views, Spark 3.2.0 added constraints on views, resulting in errors when creating views and performing JOIN queries.

Solution

To maintain consistency with versions earlier than Spark 3.2.0, the issue can be resolved by setting the client parameter spark.sql.legacy.allowAutoGeneratedAliasForView to true.