Export to Snowflake fails with an error Failed during pipeline batch build : [TASK_EXECUTION_ERROR: 2524]
When we look at the actual YARN log for the YARN application that was submitted for the export job or the Databricks log for the export JOB we may see a below error
Caused by: net.snowflake.client.jdbc.SnowflakeSQLException: Statement reached its statement or warehouse timeout of 3,600 second(s) and was canceled.
This is because the snowflake query is getting timed out after 3600 seconds
This happens because of parameter STATEMENT_TIMEOUT_IN_SECONDS set at the warehouse level with the xxx seconds.
If the warehouse level number is less for the parameter STATEMENT_TIMEOUT_IN_SECONDS, it takes precedence over the account, and therefore the query times out.
You can identify the parameter details and it's value by running the below statements and make the changes to the parameter STATEMENT_TIMEOUT_IN_SECONDS to a high value to avoid this issue :
show parameters like '%STATEMENT_TIMEOUT_IN_SECONDS%' for warehouse <warehousename>; show parameters like '%STATEMENT_TIMEOUT_IN_SECONDS%';
alter warehouse <warehousename> set STATEMENT_TIMEOUT_IN_SECONDS=172800;