Problem Description:


While replicating the hive warehouse data from the ADLS file system to Azure WASB storage, the Replicator workflow will be successful but the target hive table will not be created in the destination cluster.


Root cause:


The replicator copies the data from ADLS to Azure WASB during the migration process and it does not preserve the modified time of the file after copying the data to Azure WASB. While creating the hive table in the destination cluster replicator checks if the modified time for the data files is the same on both the source and a destination cluster, if not it will skip creating the hive table in the destination cluster. 


The hive table creating task is getting skipped as the modified time for the data files will not be the same in this case for both source and destination data files.



Solution:


Set the below property in the Replicator workflow Advanced properties as shown below. The replicator will not check the modified time for the data files if we set the below property and the hive table will be created in the destination cluster.


key: infoworks#replicator#copy#sync_modified_times

value: false


Applicable Replicator versions:

All Versions