Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Not A Bug
-
Affects Version/s: 0.2
-
Fix Version/s: None
-
Component/s: RSC
-
Labels:None
Description
There's a backward incompatible signature change in Kyro 2.22: https://github.com/twitter/chill/issues/209
It's causing errors when Spark/Hive call Kyro. An example:
An error occurred while calling o851.showString.
: java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
at org.apache.hadoop.hive.ql.exec.Utilities$3.initialValue(Utilities.java:1097)
at org.apache.hadoop.hive.ql.exec.Utilities$3.initialValue(Utilities.java:1089)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:160)
at java.lang.ThreadLocal.get(ThreadLocal.java:150)
at org.apache.spark.sql.hive.HiveShim$HiveFunctionWrapper.serializePlan(HiveShim.scala:155)
at org.apache.spark.sql.hive.HiveShim$HiveFunctionWrapper.writeExternal(HiveShim.scala:168)
Should we downgrade from 2.22 to 2.21 since Spark is using 2.21?