Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.18.0
    • Fix Version/s: None
    • Component/s: Examples
    • Labels:
      None
    • Environment:
      5.2 VM (note that this issue existed on 5.1 VM, as well)

      Description

      The example fails in two places when I try to run it on the 5.2 VM. I'm able to clone kite-examples, run mvn install, and run CreateEvents successfully.

      ```
      mvn exec:java -Dexec.mainClass="org.kitesdk.examples.spark.CreateEvents"
      ```

      I attempt to run the Spark job using this command.

      ```
      spark-submit -class org.kitesdk.examples.spark.CorrelateEvents --jars $(mvn dependency:build-classpath | grep -v '^[' | sed -e 's/:/,/g') target/kite-spark-demo*.jar
      ```

      1. The first time through, the Maven build is unable to find CorrelateEvents.class. Even though it fails, the class is built and it does appear in the target JAR.

      ```
      Warning: Local jar /home/cloudera/kite-examples/spark/Downloading does not exist, skipping.
      java.lang.ClassNotFoundException: org.kitesdk.examples.spark.CorrelateEvents
      ```

      Running the command a second time, it fails with a permissions error.

      ```
      Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=EXECUTE, inode="/user/spark":spark:spark:drwxr-x---
      ```

      I searched for `inode="/user/spark"` and found this post on the Cloudera Community:

      http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/5-1-Quickstart-VM-issue/td-p/16684

      It says that the permissions issue can be overcome by running the following commands.

      ```
      $ sudo su - hdfs
      $ hadoop fs -chmod -R 777 /user/spark
      ```

      After running those commands, I was able to successfully run the example. I don't know how this issue inadvertently resolved itself when I ran the example for witnesses on February 17. This is probably why the issue wasn't caught before. Somewhere, running something, the permissions are set. When setting up the fresh VM, I started both Hue and Cloudera Manager with all services running.

      So the question is, do we want to tell people to do the -chmod, then run the example twice, or should we adjust the example to handle both of these issues automagically?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ddawson Dennis Dawson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: