Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-824

cloudera-scm-agent errors fails with large mounted filesystem

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: CDH 5.8.0
    • Fix Version/s: None
    • Component/s: Cloudera Manager
    • Labels:
    • Environment:
      Ubuntu 14.04

      Description

      There seems to big an issue with the cloudera-scm-agent process where it will throw an AvroTypeException if there is a mount point on the host that has a HUGE amount of space.

      On an AWS instance I have mounted an "Elastic File System" mount point ("/shared") which is an NFS mount. EFS reports back a huge amount of space (9007199170887680 bytes) which seems to be to big for the "validate" method used before creating the cord to send back to the heartbeat agent.

      Example json blob that it is attempting to validate:

      {'request': {'host_id': '953bdfd1-0b69-4869-9759-4b4d193f2614',
      'host_stats': {'load_avg': [0.27, 0.34, 0.38],
      'mountpoint_available_space_bytes':

      {'/': 90046795776, '/data01': 902647525376, '/run': 1682300928, '/run/cloudera-scm-agent/process': 8396541952, '/run/lock': 5242880, '/run/shm': 8413761536, '/run/user': 104857600, '/shared': 9223371950988984320, '/sys/fs/cgroup': 4096}

      ,

      If I umount this file system, everything starts to work again.

      My installation of CDH is:

      Version: Cloudera Express 5.8.1 (#7 built by jenkins on 20160722-1141 git: a0886a893750079a4dc7f902be22d6f6e63b85a1)

      Java VM Name: Java HotSpot(TM) 64-Bit Server VM

      Java VM Vendor: Oracle Corporation

      Java Version: 1.8.0_101

      Server Time: Aug 17, 2016 4:12:05 PM, Coordinated Universal Time (UTC)

      Copyright © 2011-2016 Cloudera, Inc. All rights reserved.

      Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              joncoulter Jon Coulter
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: