Description:

In 5.0 and 5.0.1 split by configuration is not taking any effect during the table ingestion job. Even if the user configures split by the connection object was ignoring that. Sample connection object looks like below,

io.infoworks.saas.ingestion.core.connectors.jdbc.datacrawler.AbstractJDBCDataCrawler: AbstractJDBCOptions{ConnectionDetailsObject=ConnectionDetails{jdbcDriver='com.teradata.jdbc.TeraDriver', jdbcURL='jdbc:teradata://34.204.88.33/TMODE=ANSI,CHARSET=UTF8,database=dbc,RECONNECT_COUNT=10'}, query='select "customer_id" as "customer_id" , "order_id" as "order_id" , "order_date" as "order_date" , "ship_date" as "ship_date" , "cardnumber" as "cardnumber" , "cardtype" as "cardtype" , "product_id" as "product_id" , "quantity" as "quantity" , "cost" as "cost" , "item_total" as "item_total" , "order_month" as "order_month" from "customer_360"."customer_360_transactions"', SplitOptionsObject=SplitOptions{lowerBound='1', numberOfSplits='12', splitByKey='order_month', upperBound='12', isSplitableTable=false}}

This is a bug in 5.0 and 5.0.1. To resolve this one needs to apply the below-mentioned fix in the infoworks node.

JIRA: https://infoworks.atlassian.net/browse/IPD-15263,https://infoworks.atlassian.net/browse/IPD-15262


Steps:

In 5.0

stop all running jobs
su infoworks
cd /opt/infoworks/lib/ingestion/core/
mv core.jar /tmp/core.jar.bak
wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.0.0/patch/core.jar
sudo chown infoworks:infoworks core.jar
cd /opt/infoworks/lib/ingestion/connectors/sqlserver/target/
mv sqlserver.jar /tmp/sqlserver.jar.bak
wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.0.0/patch/sqlserver.jar
sudo chown infoworks:infoworks sqlserver.jar

In 5.0.1

stop all running jobs
su infoworks
cd /opt/infoworks/lib/ingestion/core/
mv core.jar /tmp/core.jar.bak
wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.0.1/patch/core.jar
sudo chown infoworks:infoworks core.jar (4060889628 607822 core.jar)
cd /opt/infoworks/lib/ingestion/connectors/sqlserver/target/
mv sqlserver.jar /tmp/sqlserver.jar.bak
wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.0.1/patch/sqlserver.jar
sudo chown infoworks:infoworks sqlserver.jar (361347319 16238 sqlserver.jar)


Applicable IWX versions:

IWX-5.0 and 5.0.1