PROBLEM DESCRIPTION


Please refer to this article if the metadata crawl fails with error: "Failed to perform metadata crawl"


CAUSE:


There are chances that the source database might have more than 5,000/10,000 tables/views/ synonyms. If we try to crawl all the metadata in one run, there is a high chance of that job getting killed/returned with error messages saying "Failed to perform metadata crawl". This is due to the fact that DB connections would be reset after some specific time interval.


WORKAROUND/ RESOLUTION


Please follow the steps provided below in order to overcome this issue,

  1. Login to Infoworks ADE UI
  2. Navigate to the source page--> source name --> Click on the Configurations in the table list page--> Modify Configurations                                  This would take you to "Add New Config" window
  3. Add the below mentioned configurations here, 
    meta_crawl_connection_threshold  : 1000
    METADATA_CRAWL_BATCH_SIZE        : 1000
    enable_autofill_fk_relations     : false
    disable_additional_table_info    : true

    PS: All the variable names are Case Sensitive.

  4. After the above settings, the metadata crawl should work fine.