Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-2894

[spark] Kill session when it fails to start

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.9.0
    • Fix Version/s: 3.10.0
    • Component/s: con.spark
    • Labels:
      None

      Description

      One way to repro and make it, forgetting to set SPARK_HOME

      15/08/10 11:16:31 WARN security.UserGroupInformation: PriviledgedActionException as:romain (auth:PROXY) via romain (auth:SIMPLE) cause:java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory
      Exception in thread "main" java.io.IOException: Cannot run program "sparkR": error=2, No such file or directory
      	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
      	at com.cloudera.hue.livy.repl.sparkr.SparkRSession$.create(SparkRSession.scala:44)
      	at com.cloudera.hue.livy.repl.ScalatraBootstrap.init(Main.scala:107)
      	at org.scalatra.servlet.ScalatraListener.configureCycleClass(ScalatraListener.scala:67)
      	at org.scalatra.servlet.ScalatraListener.contextInitialized(ScalatraListener.scala:23)
      	at com.cloudera.hue.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
      	at com.cloudera.hue.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
      	at com.cloudera.hue.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
      	at com.cloudera.hue.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
      	at com.cloudera.hue.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
      	at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
      	at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
      	at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
      	at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
      	at com.cloudera.hue.jetty.server.Server.start(Server.java:387)
      	at com.cloudera.hue.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
      	at com.cloudera.hue.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
      	at com.cloudera.hue.jetty.server.Server.doStart(Server.java:354)
      	at com.cloudera.hue.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at com.cloudera.hue.livy.WebServer.start(WebServer.scala:64)
      	at com.cloudera.hue.livy.repl.Main$.main(Main.scala:78)
      	at com.cloudera.hue.livy.repl.Main.main(Main.scala)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:666)
      	at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:160)
      	at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:158)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:158)
      	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:203)
      	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:118)
      	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      Caused by: java.io.IOException: error=2, No such file or directory
      	at java.lang.UNIXProcess.forkAndExec(Native Method)
      	at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
      	at java.lang.ProcessImpl.start(ProcessImpl.java:130)
      	at java.lang.Proc
      

        Attachments

          Activity

            People

            • Assignee:
              erickt Erick Tryzelaar
              Reporter:
              romain Romain Rigaux
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: