General

Placeholder for KB's articles

Spark configurations on Infoworks 3.1.x and 3.3.x
In Infoworks 3.1.2, when user runs the pipeline build with spark as execution engine then a spark job is submitted to perform the transformations on the dat...
Sun, 21 Mar, 2021 at 1:05 PM
Incremental ingestion failing with error during table crawl due to path /target_hdfs_path/table_id/merged/orc does not exist
Problem Description: Incremental ingestion failing with /<target_hdfs_path>/<table_id>/merged/orc does not exists. Sample stack trace looks ...
Sun, 21 Mar, 2021 at 12:38 PM
Authentication using LDAP fails with an authentication error
Problem Description: On Infoworks if the authentication is configured to use LDAP, sometimes the user may see an authentication failure when logging thr...
Tue, 16 Mar, 2021 at 7:28 AM
Pipeline build fails with "Too many open files" error
Problem statement: When user runs a considerable number (around 50) of pipeline builds with hive execution engine in parallel, the user may find that some p...
Tue, 9 Mar, 2021 at 1:14 PM
Scheduled workflow not getting triggered on time in IWX v4.2
Problem Description: A scheduled workflow will not get triggered on time if there are three or more workflows scheduled to run at the same point in time...
Wed, 5 May, 2021 at 2:18 PM
Oracle source Ingestion fails with java.sql.SQLException: ORA-28040: No matching authentication protocol
Problem Description: Oracle source ingestion job fails with the below error in the job log in Infoworks on Databricks. Caused by: org.apache.spark.Spark...
Fri, 26 Feb, 2021 at 5:34 PM
Infoworks users authentication using LDAP takes more than a minute
Problem Description: When Infoworks authentication is configured through LDAP authentication the time taken for user to login is more than a minute. ...
Fri, 19 Feb, 2021 at 9:10 AM
Ingestion job failing with error org.apache.spark.sql.execution.OutOfMemorySparkException during the merge phase
Problem Description: Ingestion job fails with the below error in Infoworks DataFoundry 3.X,4.X, Caused by: java.util.concurrent.ExecutionException: org...
Thu, 18 Feb, 2021 at 11:15 AM
Jobs failing with error Amazon.S3.AmazonS3Exception: Part number must be an integer between 1 and 10000, inclusive
Root Cause When writing data to S3 during a multi-part upload , the maximum number of parts can be 10000 Due to this threshold the jobs fail during ...
Tue, 9 Feb, 2021 at 6:28 AM
Source import fails with internal server error
Problem Description: While importing the source JSON, sometimes you might see the internal server error in the UI like below. The actual error...
Mon, 14 Dec, 2020 at 10:17 PM