Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.14.1
-
Fix Version/s: 0.15.0
-
Component/s: None
-
Labels:None
Description
The CLI create command opens a schema and adds it to the DatasetDescriptor using the InputStream method. This should detect when the schema is in HDFS and use the Hive URI property.
The literal schema property value is limited to the size of the TABLE_PARAMS value, which is about 4k for some back-ends. This results in the following error when inserting:
2014-06-10 10:17:17 INFO :: Creating a managed Hive table named: test_table
Unknown error
org.kitesdk.data.DatasetRepositoryException: Hive MetaStore exception
at org.kitesdk.data.hcatalog.impl.HCatalog.createTable(HCatalog.java:153)
at org.kitesdk.data.hcatalog.HCatalogManagedMetadataProvider.create(HCatalogManagedMetadataProvider.java:71)
at org.kitesdk.data.hcatalog.HCatalogDatasetRepository.create(HCatalogDatasetRepository.java:64)
at org.kitesdk.cli.commands.CreateDatasetCommand.run(CreateDatasetCommand.java:75)
at org.kitesdk.cli.Main.run(Main.java:133)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.kitesdk.cli.Main.main(Main.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: MetaException(message:javax.jdo.JDODataStoreException: Put request failed : INSERT INTO "TABLE_PARAMS" ("PARAM_VALUE","TBL_ID","PARAM_KEY") VALUES (?,?,?)
NestedThrowables:
org.datanucleus.store.rdbms.exceptions.MappedDatastoreException: INSERT INTO "TABLE_PARAMS" ("PARAM_VALUE","TBL_ID","PARAM_KEY") VALUES (?,?,?) )
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_result$create_table_resultStandardScheme.read(ThriftHiveMetastore.java:20149)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_result$create_table_resultStandardScheme.read(ThriftHiveMetastore.java:20117)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_result.read(ThriftHiveMetastore.java:20043)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:732)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:719)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:432)
at org.kitesdk.data.hcatalog.impl.HCatalog$3.call(HCatalog.java:139)
at org.kitesdk.data.hcatalog.impl.HCatalog$3.call(HCatalog.java:136)
at org.kitesdk.data.hcatalog.impl.HCatalog.doWithRetry(HCatalog.java:67)
at org.kitesdk.data.hcatalog.impl.HCatalog.createTable(HCatalog.java:145)
... 11 more