Uploaded image for project: 'Kite SDK (READ-ONLY)'
  1. Kite SDK (READ-ONLY)
  2. KITE-328

Avoid using singleton hive metastore client concurrently

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.12.1
    • Component/s: None
    • Labels:
      None

      Description

      Exceptions occur when writing to multiple partitioned writers for datasets in repo:hive repositories.
      These exceptions are caused because HCatUtil.getHiveClient(...) (in kite's HCatalog class) can return the same instance in different threads.

      Example stacktrace:
      writer 1 writing record 0
      writer 2 writing record 0
      14/02/18 16:28:57 INFO metastore.HiveMetaStore: 1: source:/127.0.0.1 append_partition_by_name: db=default tbl=test2 part=username=48
      14/02/18 16:28:57 INFO HiveMetaStore.audit: ugi=karel ip=/127.0.0.1 cmd=source:/127.0.0.1 append_partition_by_name: db=default tbl=test2 part=username=48
      14/02/18 16:28:57 INFO metastore.HiveMetaStore: 1: source:/127.0.0.1 append_partition_by_name: db=default tbl=test3 part=username=48
      14/02/18 16:28:57 INFO HiveMetaStore.audit: ugi=karel ip=/127.0.0.1 cmd=source:/127.0.0.1 append_partition_by_name: db=default tbl=test3 part=username=48
      java.lang.IllegalArgumentException: Problem creating view for entity:

      {"username": "0", "email": "0@example.com"}

      at org.kitesdk.data.filesystem.PartitionedDatasetWriter.write(PartitionedDatasetWriter.java:96)
      at org.kitesdk.data.hcatalog.TestExternalHCatalogDatasetRepository$WriteRecordTask.call(TestExternalHCatalogDatasetRepository.java:253)
      at org.kitesdk.data.hcatalog.TestExternalHCatalogDatasetRepository$WriteRecordTask.call(TestExternalHCatalogDatasetRepository.java:234)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:701)
      Caused by: java.lang.RuntimeException: Hive metastore exception
      at org.kitesdk.data.hcatalog.HCatalog.addPartition(HCatalog.java:108)
      at org.kitesdk.data.hcatalog.HCatalogMetadataProvider.partitionAdded(HCatalogMetadataProvider.java:94)
      at org.kitesdk.data.filesystem.PartitionedDatasetWriter$DatasetWriterCacheLoader.load(PartitionedDatasetWriter.java:179)
      at org.kitesdk.data.filesystem.PartitionedDatasetWriter$DatasetWriterCacheLoader.load(PartitionedDatasetWriter.java:154)
      at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
      at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
      at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
      at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
      at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
      at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
      at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
      at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4834)
      at org.kitesdk.data.filesystem.PartitionedDatasetWriter.write(PartitionedDatasetWriter.java:94)
      ... 7 more
      Caused by: org.apache.thrift.TApplicationException: append_partition_by_name failed: out of sequence response
      at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_by_name(ThriftHiveMetastore.java:1165)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_by_name(ThriftHiveMetastore.java:1150)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:399)
      at org.kitesdk.data.hcatalog.HCatalog.addPartition(HCatalog.java:102)
      ... 19 more

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              karel1980 Karel Vervaeke
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: