When writing data to S3 during a multi-part upload , the maximum number of parts can be 10000
Due to this threshold the jobs fail during execution
To overcome this limitation we need to increase the size of the ‘part’ with the below advanced configuration.
The part size can be up to 5 GB
Set the below property as an additional configuration in table properties as shown in below image. This will increase the capacity for each 'part'
Please rerun the job after setting the above advanced configuration.
IWX 3.2,4.2 and above