Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.8.1
-
Fix Version/s: maintenance-tasks
-
Component/s: Data Module
-
Labels:None
Description
Joe T. reported a bug where the default avro ClassLoader doesn't match the ClassLoader for the jar where his specific class definitions are:
Thread.currentThread().getContextClassLoader(); // ==> java.net.URLClassLoader@11c5ed81 <-- can load his avro-specific classes SpecificData.get().getClassLoader().toString(); // ==> sun.misc.Launcher$AppClassLoader@111f2041 <-- avro instantiates GenericData.Record (can't find specific classes) this.getClass().getClassLoader(); // ==> java.net.URLClassLoader@11c5ed81
Joe is using hadoop jar to run the reader command. I've suggested he separate the classes from his main jar and add them using the HADOOP_CLASSPATH env variable as a work-around.
CDK needs to allow the user to set the ClassLoader avro uses to find the specific classes, rather than relying on the default instance.
Attachments
Issue Links
- depends on
-
KITE-496 Datasets#load, create, and update should take in a Class<E> parameter so you can control the type of the entities you want
-
- Resolved
-