Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-312

TaskTracker Out of Memory because of distributed cache

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: CDH3u2
    • Fix Version/s: CDH3u3
    • Component/s: MapReduce
    • Labels:
      None

      Description

      This Out of Memory happens when you run large number of jobs (using the distributed cache) to a TaskTracker. Seems the basic issue is with the distributedCacheManager (instance of TrackerDistributedCacheManager in TaskTracker.java), this gets created during TaskTracker.initialize(), and it keeps references to TaskDistributedCacheManager for every submitted job via the jobArchives Map, also references to CacheStatus via cachedArchives map. I am not seeing these cleaned up between jobs, so this can cause memory problems when really large number of jobs are submitted. I am suspecting that this is the cause for the particular problem discussed in this thread: https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/thread/e678a3e85ef95beb/157aaa94a5a901d8?pli=1#157aaa94a5a901d8

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                eli Eli Collins
                Reporter:
                ahmed Ahmed Radwan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: