Details
-
Type: Bug
-
Status: Open
-
Priority: Major
-
Resolution: Unresolved
-
Affects Version/s: 0.18.0
-
Fix Version/s: None
-
Component/s: Examples
-
Labels:None
-
Environment:5.2 VM (note that this issue existed on 5.1 VM, as well)
Description
The example fails in two places when I try to run it on the 5.2 VM. I'm able to clone kite-examples, run mvn install, and run CreateEvents successfully.
```
mvn exec:java -Dexec.mainClass="org.kitesdk.examples.spark.CreateEvents"
```
I attempt to run the Spark job using this command.
```
spark-submit -class org.kitesdk.examples.spark.CorrelateEvents --jars $(mvn dependency:build-classpath | grep -v '^[' | sed -e 's/:/,/g') target/kite-spark-demo*.jar
```
1. The first time through, the Maven build is unable to find CorrelateEvents.class. Even though it fails, the class is built and it does appear in the target JAR.
```
Warning: Local jar /home/cloudera/kite-examples/spark/Downloading does not exist, skipping.
java.lang.ClassNotFoundException: org.kitesdk.examples.spark.CorrelateEvents
```
Running the command a second time, it fails with a permissions error.
```
Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=EXECUTE, inode="/user/spark":spark:spark:drwxr-x---
```
I searched for `inode="/user/spark"` and found this post on the Cloudera Community:
http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/5-1-Quickstart-VM-issue/td-p/16684
It says that the permissions issue can be overcome by running the following commands.
```
$ sudo su - hdfs
$ hadoop fs -chmod -R 777 /user/spark
```
After running those commands, I was able to successfully run the example. I don't know how this issue inadvertently resolved itself when I ran the example for witnesses on February 17. This is probably why the issue wasn't caught before. Somewhere, running something, the permissions are set. When setting up the fresh VM, I started both Hue and Cloudera Manager with all services running.
So the question is, do we want to tell people to do the -chmod, then run the example twice, or should we adjust the example to handle both of these issues automagically?