Target Audience: All Infoworks Product Users


List of Steps/Process followed by Infoworks in Metadata Crawl process


1.Infoworks-Product gets the list of the DB objects(tables, views etc) by using the JDBC provided method to get the metadata of the source.


2.Then we loop through the tables get the following additional information for the table.

 

  • Column names and data types: This is done by querying the connection with SQL "select * from table where 1=0". This will result in the empty resultset and then we use the metadata of the resultset to get the column names and data types.

  • For each table, we also query for primary keys, foreign keys, relations, and indexes. But querying for these is configurable.

3. While running the loop we renew (drop and reconnect) the DB connection after every X tables. X is configurable.


4. A mongo entry is created for every table. These entries are flushed to mongo in batches of size Y. Y is configurable.


Thanks,

Sri