Hbase mapreduce jobs (bulk import) fail due on duplicate MANIFEST.MF in jars generated by hadoop

It seams like the HBase mapreduce functionality (HFileOutputFormat) broke in the final version of cdh4. This issue did not exist in cdh 4.0.0b2; it only occurred after upgrading to final. It now fails when the job tries to generate on the fly jar, and it fails with a duplicate MANIFEST.MF file.

This seams to be causes by org.apache.hadoop.hbase.mapreduce.findOrCreateJar and the class that gets passed into is is: org.apache.hadoop.hbase.io.ImmutableBytesWritable.

Exception in thread "main" java.io.IOException: java.lang.RuntimeException: java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:521)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:472)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:438)
at org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:341)
at com.adfin.hadoop.jobs.csvimport.HFileImport.createJob(HFileImport.java:41)
at com.adfin.hadoop.jobs.csvimport.HFileImport.main(HFileImport.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.RuntimeException: java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
at org.apache.hadoop.util.JarFinder.getJar(JarFinder.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:518)
... 10 more
Caused by: java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:175)
at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:92)
at org.apache.hadoop.util.JarFinder.zipDir(JarFinder.java:66)
at org.apache.hadoop.util.JarFinder.zipDir(JarFinder.java:62)
at org.apache.hadoop.util.JarFinder.zipDir(JarFinder.java:44)
at org.apache.hadoop.util.JarFinder.createJar(JarFinder.java:93)
at org.apache.hadoop.util.JarFinder.getJar(JarFinder.java:134)
... 15 more