Write partitioning properties#
The property described in this page is defined as follows:
Kubernetes: In the
additionalProperties
section of the the top-levelcoordinator
andworker
nodes in thevalues.yaml
file.
use-preferred-write-partitioning
#
Type: boolean
Default value:
true
Session property:
use_preferred_write_partitioning
Enable preferred write partitioning. When set to true
, each partition is
written by a separate writer. For some connectors such as the Hive connector,
only a single new file is written per partition, instead of multiple files.
Partition writer assignments are distributed across worker nodes for parallel
processing.