Details
Major Description
For instance, SHORT doesn't work and the following:
register recordservice-hcatalog-pig-adapter-0.3.0-cdh5.7.x-jar-with-dependencies.jar
A = LOAD 'tpch.nation' USING com.cloudera.recordservice.pig.HCatRSLoader();
DUMP A;
gives error:
2016-03-11 16:14:13,459 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1457726789235_0134 2016-03-11 16:14:13,459 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases A 2016-03-11 16:14:13,459 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: A[4,4] C: R: 2016-03-11 16:14:13,532 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 2016-03-11 16:14:46,248 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete 2016-03-11 16:14:48,919 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2016-03-11 16:14:48,919 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1457726789235_0134 has failed! Stop running all dependent jobs 2016-03-11 16:14:48,920 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 2016-03-11 16:14:49,070 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR: Unexpected data type java.lang.Short found in stream. Note only standard Pig type is supported when you output from UDF/LoadFunc 2016-03-11 16:14:49,070 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! 2016-03-11 16:14:49,070 [main] WARN org.apache.pig.tools.pigstats.ScriptState - unable to read pigs manifest file 2016-03-11 16:14:49,072 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: HadoopVersion PigVersion UserId StartedAt FinishedAt Features 2.6.0-cdh5.7.0-SNAPSHOT chao.sun 2016-03-11 16:13:48 2016-03-11 16:14:49 UNKNOWN Failed! Failed Jobs: JobId Alias Feature Message Outputs job_1457726789235_0134 A MAP_ONLY Message: Job failed! hdfs://vd0214.halxg.cloudera.com:8020/tmp/temp-1087459412/tmp771951061, Input(s): Failed to read data from "tpch.nation" Output(s): Failed to produce result in "hdfs://vd0214.halxg.cloudera.com:8020/tmp/temp-1087459412/tmp771951061" Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager spill count : 0 Total bags proactively spilled: 0 Total records proactively spilled: 0 Job DAG: job_1457726789235_0134 2016-03-11 16:14:49,072 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! 2016-03-11 16:14:49,102 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias A. Backend error : Unexpected data type java.lang.Short found in stream. Note only standard Pig type is supported when you output from UDF/LoadFunc
while without RS this works.
I think on the RS side we need to do some type collapsing to prevent this from happening.