Configurable Wait time and # of Retries for querying Infoworks MR job status


It was observed that, in some cases, merge job would occasionally hang for secondary partitions because the RPC calls for job completion status would not return for a single thread, which would, in turn, block other threads. To resolve this issue, it was recommended that instead of waiting indefinitely, Infoworks should wait for a configurable amount of time and retry again. The number of retries should also be configurable. These configurations are now available to be set at the Admin level.

Configurations:

IW_JOB_STATUS_TIMEOUT_SECONDS: The number of seconds to wait for a response from the job completion status query. Default is set to 20.


IW_JOB_STATUS_RETRIES: Number of times the job completion status query will be retried, before failing. After retrying these many times the secondary partition merge for that particular job will fail. Default is set to 3.