Эх сурвалжийг харах

conf(spark): 默认开启 hive 动态分区 nonstrict

加 hive.exec.dynamic.partition=true / mode=nonstrict 到 spark-defaults.conf,
数仓默认全动态分区写法,避免每个 SQL 顶部重复 SET。spark-submit 会对
hive.* 报 "Ignoring non-spark config" warning,是噪声不影响行为
(builder.config 注入 SparkSession SQLConf 走另一条路径)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tianyu.chu 19 цаг өмнө
parent
commit
5738e6b908

+ 5 - 0
conf/spark-defaults.conf

@@ -21,3 +21,8 @@ spark.sql.codegen.wholeStage                    false
 spark.sql.execution.arrow.enabled               true
 spark.sql.execution.arrow.fallback.enabled     true
 spark.sql.statistics.fallBackToHdfs             true
+
+# Hive 动态分区(数仓默认全动态分区写法,避免每个 SQL 顶部重复 SET;
+# spark-submit 会对 hive.* 报 "Ignoring non-spark config" warning,是噪声不影响行为)
+hive.exec.dynamic.partition                     true
+hive.exec.dynamic.partition.mode                nonstrict