Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-600

Regression in MRv2 VS MRv1 for io.sort.mb

          Details

          • Type: Bug
          • Status: Open
          • Priority: Critical
          • Resolution: Unresolved
          • Affects Version/s: CDH4.6.0
          • Fix Version/s: None
          • Component/s: MapReduce
          • Labels:
          • Environment:
            YARN (MRv2)

            Description

            Jobs running on MRv1 with io.sort.mb=2047 (the current limit) are failing miserably on MRv2 due to an overflow bug in
            hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
            (which was fixed in MRv1 during the CDH3 to CDH4 transition)

            The bug is fixed in CDH5, but is probably easy enough to be included in CDH4.7.0

              Attachments

                Activity

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    jbnote Jean-Baptiste Note
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated: