Please refer to this article if the metadata crawl fails with error: "Failed to perform metadata crawl"
There are chances that the source database might have more than 5,000/10,000 tables/views/ synonyms. If we try to crawl all the metadata in one run, there is a high chance of that job getting killed/returned with error messages saying "Failed to perform metadata crawl". This is due to the fact that DB connections would be reset after some specific time interval.
Please follow the steps provided below in order to overcome this issue,
- Login to Infoworks ADE UI
- Navigate to the source page--> source name --> Click on the Configurations in the table list page--> Modify Configurations This would take you to "Add New Config" window
- Add the below mentioned configurations here,
meta_crawl_connection_threshold : 1000 METADATA_CRAWL_BATCH_SIZE : 1000 enable_autofill_fk_relations : false disable_additional_table_info : true
PS: All the variable names are Case Sensitive.
After the above settings, the metadata crawl should work fine.