General

Placeholder for KB's articles

Optimizing Ingestion Performance and Reducing Cluster Costs Using Split-By and Table Group Parallelism in Infoworks
Issue Summary Users may observe slow ingestion performance and higher compute costs when ingesting large tables or many tables in a single job. This is oft...
Fri, 2 May, 2025 at 7:23 AM
Databricks Cluster Launch Fails Due to Cluster Policy Mismatch
Issue Summary When triggering an ingestion job in Infoworks using a Databricks Ephemeral Cluster, the cluster launch fails with a policy validation error. ...
Fri, 2 May, 2025 at 7:14 AM
"Test Connection" Fails from Infoworks to Data Source
Issue Summary When testing a connection to a data source from Infoworks, users may observe that the Test Connection fails, even though the network settings...
Tue, 15 Apr, 2025 at 11:29 AM
Troubleshooting Insufficient Privileges for Snowflake Roles
Issue Summary Customers may encounter errors when running queries through the platform on Snowflake. A common root cause is insufficient privileges granted...
Mon, 14 Apr, 2025 at 11:36 AM
Triggering Databricks Notebook from Infoworks Bash Node
Part 1: Creating a Databricks Job to Run a Notebook 1: Create the Notebook in Databricks Open your Databricks workspace. Navigate to the Workspace secti...
Mon, 14 Apr, 2025 at 12:57 PM
Custom docker image
Check if Docker Installed: Ensure that Docker is installed on your system. You can verify this by running: docker --version Create Dockerfil...
Mon, 14 Apr, 2025 at 12:58 PM
Common infrastructure issues during Infoworks Installation on AKS
Issue 1: Pods fails to come up with CreateContainerConfigError. Error Message: failed to prepare subPath for volumeMount "dit-diyprod-rel-a-logs"...
Wed, 29 May, 2024 at 6:14 PM
IBM DB2 ingestion job fails with CharConversionException
Problem Character encoding issues can sometimes cause errors during data ingestion from an IBM DB2 database. These errors are typically related to data in ...
Thu, 2 Nov, 2023 at 2:55 AM
Pipeline Job failing with Dates older than '1582-10-15' or Timestamp older than ‘1900-01-01 00:00:00.0'
This is due to Spark 2.X's legacy calendar. Please add the below Spark configuration to your pipeline job and retry the job Key:iw_spark_app_co...
Fri, 22 Sep, 2023 at 9:19 AM
Write to BQ jobs, Direct load and Indirect load options
After running several internal tests with both the Direct load and Indirect load options, we have observed the following performance results: Please no...
Tue, 22 Aug, 2023 at 11:29 PM