Details
-
Type:
Sub-task
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 0.4.0
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
Description
Error running Pig jobs over Hcatalog tables sourced from parquet files with schema different than the table schema. For e.g. the table schema is defined as string,string,double ​but the source parquet file has schema defined as string,double,double.
Caused by: TRecordServiceException(code:INTERNAL_ERROR, message:Task failed due to an internal error., detail:File 'hdfs://filePath/fileName.parquet' has an incompatible Parquet schema for column 'columnName'. Column type: STRING, Parquet schema: required double rate [i:1 d:0 r:0] ) at com.cloudera.recordservice.thrift.RecordServiceWorker$Fetch_result$Fetch_resultStandardScheme.read(RecordServiceWorker.java:2911) at com.cloudera.recordservice.thrift.RecordServiceWorker$Fetch_result$Fetch_resultStandardScheme.read(RecordServiceWorker.java:2888) at com.cloudera.recordservice.thrift.RecordServiceWorker$Fetch_result.read(RecordServiceWorker.java:2827) at com.cloudera.recordservice.shade.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at com.cloudera.recordservice.thrift.RecordServiceWorker$Client.recv_Fetch(RecordServiceWorker.java:155) at com.cloudera.recordservice.thrift.RecordServiceWorker$Client.Fetch(RecordServiceWorker.java:142) at com.cloudera.recordservice.core.RecordServiceWorkerClient.fetch(RecordServiceWorkerClient.java:343)