Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: CDH 5.4.4
-
Fix Version/s: None
-
Component/s: MapReduce
-
Labels:None
Description
We are using the capacity scheduler and MapReduce v1. After we updated to CDH 5.4.4, the job tracker is not starting and we get the following error:
2015-07-24 08:59:37,223 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.CapacityTaskScheduler not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2144) at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1848) at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1757) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:305) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:297) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4554) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.CapacityTaskScheduler not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2136) ... 5 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.CapacityTaskScheduler not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110) ... 6 more
We think that there is a bug in the HADOOP_CLASSPATH, since there is no ":" separator between both capacity-scheduler jar files:
STARTUP_MSG: classpath = [...]:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hadoop-0.20-mapreduce/contrib/capacity-scheduler/hadoop-capacity-scheduler-2.6.0-mr1-cdh5.4.4.jar/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hadoop-0.20-mapreduce/contrib/capacity-scheduler/hadoop-capacity-scheduler-mr1.jar:[...]
As a workaround we added the jar files manually to the classpath and our cluster is now running.