Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-706

mapreduce.job.counters.max not read by MRAppMaster

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: CDH 5.3.2
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None
    • Environment:
      Linux Ubuntu 12.04, AWS

      Description

      When running a MR-job on YARN cluster with more than 120 counters and attempting to view the counters via web-interface, internal server error is thrown. In addition to that, when MRAppMaster is about to collect counters from finished tasks, it fails with LimitExceededException complaining about the same. Message is
      "Too many counters: 121 max=120" despite of settings in mapred-site.xml. In may case mapreduce.job.counters.max = 65536.

      Release notes for 5.3.2 (http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.5.0-cdh5.3.2.releasenotes.html) says that release contains a fix for [MAPREDUCE-5875] (https://issues.apache.org/jira/browse/MAPREDUCE-5875) but in fact there is 0160-CLOUDERA-BUILD.-CDH-19529-Precommit-version-of-MAPRE.patch file in cloudera/patches which modifies only org.apache.hadoop.mapred.JobClient
      and similar patch for mr1. After applying patch for MRAppMaster, problem is gone.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              whale2 Nikita Makeev
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: