Problem Description


Incremental job for a csv source fails with the below error in the log


14:36:33,618 ERROR MergeUtils:112 - java.lang.AssertionError: connection_string for job is not specified
at infoworks.discovery.incremental.PostMergeSwitch.getDBConn(PostMergeSwitch.java:537)
at infoworks.discovery.incremental.PostMergeSwitch.getVaidator(PostMergeSwitch.java:524)
at infoworks.discovery.incremental.PostMergeSwitch.switchTable(PostMergeSwitch.java:341)
at infoworks.discovery.incremental.CDCParallelMerge.trySwitch(CDCParallelMerge.java:550)
at infoworks.discovery.incremental.CDCParallelMerge.postJob(CDCParallelMerge.java:538)
at infoworks.discovery.incremental.CDCParallelMerge.runMerge(CDCParallelMerge.java:412)
at infoworks.discovery.incremental.CDCParallelMerge.runMerge(CDCParallelMerge.java:299)
at infoworks.discovery.incremental.CDCParallelMerge.runMerge(CDCParallelMerge.java:281)
at infoworks.discovery.filecrawler.generic.Crawler.runMerge(Crawler.java:1600)
at infoworks.discovery.filecrawler.generic.Crawler.runCDCMerge(Crawler.java:1617)
at infoworks.discovery.filecrawler.generic.Crawler.crawl(Crawler.java:518)
at infoworks.discovery.filecrawler.generic.Crawler$1.call(Crawler.java:504)
at infoworks.discovery.filecrawler.generic.Crawler$1.call(Crawler.java:501)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)


Root cause


This issue is specific to CVS (Patch hotfix/10008 and IWX version 2.2.2). It occurs during a Batch-ID-based Incremental load. We do a validation for DB connection properties for RDBMS sources and we skip this validation for File based sources. In the patch hotfix/10008, we have excluded the validation only for FileBasedMergeReducer but we need to exclude the validation for BatchIdBasedRedcer as well. This issue happens for Batch-ID-based Incremental load as we did not exclude the validation for it.


Solution


Set the value to false in the Admin Configurations for the below constant at the source level and run the job again. This should resolve the issue.


SOURCE_LVL_MERGE_VALIDATION_ENABLED   


value is false


IWX version 2.2.2/hotfix/10008


-Aditya