Problem Description


Oracle ingestion job fails with the below container error during the mapreduce job


Container [pid=53100,containerID=container_e67_1525266108447_164209_01_000002] is running beyond physical memory limits. Current usage: 2.3 GB of 2 GB physical memory used; 3.8 GB of 4.2 GB virtual memory used. Killing container. 


Root cause


This issue occurs if the Mapper/reducer job fails out of memory


Solution


You could increase the mapper and reducer memory for the MR job by increasing the values for the below properties in $IW_HOME/conf/conf.properties file


iw_jobs_default_mr_map_mem_mb=2048

iw_jobs_default_mr_map_mem_mb=2048


Default value for the above properties would be 2gb(2048). You could increase it further say 4gb(4096) and start the job again.


If it is already set to 4GB, increase it to 6 GB and then run the job


IWX version: 2.3.x 


--Aditya