General

Placeholder for KB's articles

Ingestion job failing with error org.apache.spark.sql.execution.OutOfMemorySparkException during the merge phase
Problem Description: Ingestion job fails with the below error in Infoworks DataFoundry 3.X,4.X, Caused by: java.util.concurrent.ExecutionException: org...
Thu, 18 Feb, 2021 at 11:15 AM
Jobs failing with error Amazon.S3.AmazonS3Exception: Part number must be an integer between 1 and 10000, inclusive
Root Cause When writing data to S3 during a multi-part upload , the maximum number of parts can be 10000 Due to this threshold the jobs fail during ...
Tue, 9 Feb, 2021 at 6:28 AM
Source import fails with internal server error
Problem Description: While importing the source JSON, sometimes you might see the internal server error in the UI like below. The actual error...
Mon, 14 Dec, 2020 at 10:17 PM
Rest API requests return an error ‘Failed to retrieve username and password from auth token’
Problem Description When an API request is made to the Infoworks platform using an auth token, the response request returns an error ‘Failed to retrieve...
Mon, 14 Dec, 2020 at 10:12 PM
How to pass yarn queue to run a hive sql in a workflow bash script node
Description You could pass the yarn queue in the bash script task to make sure the hive query that you execute runs in that particular yarn queue. P...
Tue, 8 Dec, 2020 at 1:40 PM
Rabbit MQ fails to start
Problem Description   Rabbit MQ fails to start when restarting Infoworks services   An error like the below can be seen during the restart of Infoworks s...
Fri, 13 Nov, 2020 at 10:44 PM
The pipeline build failing with java.lang.OutOfMemoryError: GC overhead limit exceeded
Description: The pipeline builds failing with OutOfMemoryError: GC overhead limit exceeded error. Sample error log looks like below, 20/10/13 13:30:...
Thu, 12 Nov, 2020 at 3:44 PM
OutOfMemorySparkException while running Infoworks Jobs
Problem Description When you are running some incremental ingestion jobs or pipeline jobs, sometimes you might see the below errors related to broadcast...
Mon, 9 Nov, 2020 at 9:28 PM
The pipeline build failing with "Exception: cannot resolve (column name) given input columns" error
Description: The pipeline builds failing with cannot resolve 'some column name' given input columns error. Sample error log looks like below, ...
Fri, 30 Oct, 2020 at 11:28 AM
Ingestion job TIMEDOUT after running for long time
Databricks jobs, for ingestion, submitted by infoworks DF can get timed out if it runs more than a configurable limit. The messages in job log which indicat...
Fri, 30 Oct, 2020 at 10:53 AM