Details
Major Description
Livy unit test can not pass with spark 1.5 due to one issue in snappy-java 1.0.4.1 which is shipped with spark 1.5.
Tests in error: TestSparkClient.testAddJarsAndFiles:175->runTest:434 » Execution java.lang.Run... TestSparkClient.testHiveJob:244->runTest:434 » Execution java.lang.RuntimeExce... TestSparkClient.testRemoteClient:164->runTest:434 » Execution java.lang.Runtim... TestSparkClient.testSimpleSparkJob:115->runTest:434 » Execution java.lang.Runt... TestSparkClient.testSparkSQLJob:230->runTest:434 » Execution java.lang.Runtime...
Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1865)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
... 50 more
Specify spark 1.6 when running unit test can solve this issue. And since spark 1.6.1 has been released for quite a while, I think we can upgrade the default spark version to 1.6.1 in pom
Attachments
Issue Links
- links to