Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-759

Null Pointer exception while submitting Mapreduce job from Eclipse (windows platform) to remote cluster(unix platform)

    Details

    • Type: Escalation
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None

      Description

      We are building a stand alone windows application in Java which submits a Mapreduce job using runHadoopJob() method to a remote Unix cluster. Getting null pointer exception in the process.Iam using Jars of Hadoop 2.5 distribution,copied winutils.exe to the bin folder and cross platform flag is also set to true. Below is the trace of exception.

      PS : same code is running without any issues in Cloudera quick start VM.

      java.lang.NullPointerException
      at java.lang.ProcessBuilder.start(Unknown Source)
      at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)
      at org.apache.hadoop.util.Shell.run(Shell.java:455)
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:774)
      at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:646)
      at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:434)
      at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:282)
      at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133)
      at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:435)
      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Unknown Source)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
      at com.deloitte.dmart.hadoop.MapreduceDriver.runHadoopJob(MapreduceDriver.java:109)
      at com.deloitte.dmart.panels.SourceAndTargetTabbedPanel$FileTask.doInBackground(SourceAndTargetTabbedPanel.java:459)
      at com.deloitte.dmart.panels.SourceAndTargetTabbedPanel$FileTask.doInBackground(SourceAndTargetTabbedPanel.java:1)
      at javax.swing.SwingWorker$1.call(Unknown Source)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at javax.swing.SwingWorker.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rock123 Rcoky
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: