• Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.18.0
    • Fix Version/s: None
    • Component/s: Examples
    • Labels:
    • Environment:
      5.2 VM (note that this issue existed on 5.1 VM, as well)


      The example fails in two places when I try to run it on the 5.2 VM. I'm able to clone kite-examples, run mvn install, and run CreateEvents successfully.

      mvn exec:java -Dexec.mainClass="org.kitesdk.examples.spark.CreateEvents"

      I attempt to run the Spark job using this command.

      spark-submit -class org.kitesdk.examples.spark.CorrelateEvents --jars $(mvn dependency:build-classpath | grep -v '^[' | sed -e 's/:/,/g') target/kite-spark-demo*.jar

      1. The first time through, the Maven build is unable to find CorrelateEvents.class. Even though it fails, the class is built and it does appear in the target JAR.

      Warning: Local jar /home/cloudera/kite-examples/spark/Downloading does not exist, skipping.
      java.lang.ClassNotFoundException: org.kitesdk.examples.spark.CorrelateEvents

      Running the command a second time, it fails with a permissions error.

      Exception in thread "main" Permission denied: user=cloudera, access=EXECUTE, inode="/user/spark":spark:spark:drwxr-x---

      I searched for `inode="/user/spark"` and found this post on the Cloudera Community:

      It says that the permissions issue can be overcome by running the following commands.

      $ sudo su - hdfs
      $ hadoop fs -chmod -R 777 /user/spark

      After running those commands, I was able to successfully run the example. I don't know how this issue inadvertently resolved itself when I ran the example for witnesses on February 17. This is probably why the issue wasn't caught before. Somewhere, running something, the permissions are set. When setting up the fresh VM, I started both Hue and Cloudera Manager with all services running.

      So the question is, do we want to tell people to do the -chmod, then run the example twice, or should we adjust the example to handle both of these issues automagically?




            • Assignee:
              ddawson Dennis Dawson
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created: