Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 1.1.0
-
Fix Version/s: None
-
Component/s: Data Module
-
Labels:None
Description
When importing a JSON file into a dataset with a float column, I got this error:
java.lang.Exception: org.kitesdk.data.DatasetRecordException: Cannot convert field rating
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: org.kitesdk.data.DatasetRecordException: Cannot convert field rating
at org.kitesdk.data.spi.JsonUtil.convertField(JsonUtil.java:356)
at org.kitesdk.data.spi.JsonUtil.convertToAvro(JsonUtil.java:247)
at org.kitesdk.data.spi.filesystem.JSONFileReader$1.apply(JSONFileReader.java:103)
at org.kitesdk.data.spi.filesystem.JSONFileReader$1.apply(JSONFileReader.java:99)
at com.google.common.collect.Iterators$8.next(Iterators.java:812)
at org.kitesdk.data.spi.filesystem.JSONFileReader.next(JSONFileReader.java:121)
at org.kitesdk.data.spi.filesystem.JSONFileReader$JSONRecordReader.nextKeyValue(JSONFileReader.java:164)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.kitesdk.data.DatasetRecordException: Cannot resolve union: 4.0 not in ["null", "float"]
at org.kitesdk.data.spi.JsonUtil.resolveUnion(JsonUtil.java:407)
at org.kitesdk.data.spi.JsonUtil.convertToAvro(JsonUtil.java:274)
at org.kitesdk.data.spi.JsonUtil.convertField(JsonUtil.java:348)
... 18 more
1 job failure(s) occurred:
It appears that JSON parsed the value as a double, which wasn't coerced to a float even though it would fit.