Uploaded image for project: 'CDH'
  1. CDH
  2. DISTRO-726

Regression: NullPointerException when writing to bucketed tables in Hive

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: CDH 5.4.2
    • Fix Version/s: None
    • Component/s: Hive
    • Labels:
      None

      Description

      We hit a regression when upgrading from CDH 5.3.3 to CDH 5.4.2. Queries which use CLUSTERED BY tables are failing with a null pointer exception:

      2015-05-26 01:26:35,729 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{},"value":{"_col0":235015151,"_col1":10,"_col2":3}}
      	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:265)
      	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
      	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
      	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{},"value":{"_col0":235015151,"_col1":10,"_col2":3}}
      	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:253)
      	... 7 more
      Caused by: java.lang.NullPointerException
      	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.findWriterOffset(FileSinkOperator.java:761)
      	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:689)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
      	at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
      	at org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
      	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:244)
      	... 7 more
      
      

      We tracked the issue down to https://issues.apache.org/jira/browse/HIVE-10538 . Applying that patch in a local build of the CDH Hive RPMs fixed the issue for us.

      Any chance of this being rolled into the next CDH update?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sveiss Stephen Veiss
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: