Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-238

Not able to project into 'group' tuple from FILTER (PIG) -- See PIG-2087

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: CDH3u0
    • Fix Version/s: CDH3u1
    • Component/s: Pig
    • Labels:
      None

      Description

      Filed from https://issues.apache.org/jira/browse/PIG-2087 as Pig contributors cannot reproduce on Pig 0.8.

      GROUP creates the 'group' tuple, but subsequent FILTER statements cannot project into it without throwing ClassCastExceptions.

      Example:

      -----------

      my_data = LOAD 'test.txt' using PigStorage(',')
      as (name:chararray, age:int, eye_color:chararray, height:int);

      by_age_and_color = GROUP my_data BY (age, eye_color);

      OUT2 = FILTER by_age_and_color by group.age is not null;
      dump OUT2;

      • I get a similar problem even if I do something like:

      OUT3 = FILTER by_age_and_color by group.age > 9;
      dump OUT3;

      --------- sample test.txt ---------
      ravi,33,blue,43
      brendan,33,green,53
      ravichandra,15,blue,43
      leonor,15,brown,46
      caeser,18,blue,23
      JCVD,,blue,23
      anthony,33,blue,46
      xavier,23,blue,13
      patrick,18,blue,33
      sang,33,brown,44

      -------------------------------

      java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)

        Attachments

          Activity

            People

            • Assignee:
              phunt Patrick Hunt
              Reporter:
              doeklund Daniel Eklund
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: