Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-684

Potential memory leak with while using Flume AsyncHbaseSink

          Details

          • Type: Bug
          • Status: Open
          • Priority: Critical
          • Resolution: Unresolved
          • Affects Version/s: CDH4.7.0
          • Fix Version/s: None
          • Component/s: Flume
          • Labels:
            None

            Description

            In our Flume configuration we are using AsyncHbaseSink as one of the sinks. Periodically we are seeing Flume hitting it's heap limit (stuck in GC). This normally happens after we start seeing timeout messages.Flume heap dumps are attached.

            Open the memory dumps in MemoryAnalyzer. Most of the preprocessing is already done on the attached heap dump. Flume logs are attached as well.

            2015-01-14 12:19:42,941 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.
            org.apache.flume.EventDeliveryException: Could not write events to Hbase. Transaction failed, and rolled back.
            at org.apache.flume.sink.hbase.AsyncHBaseSink.process(AsyncHBaseSink.java:294)
            at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
            at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
            at java.lang.Thread.run(Thread.java:744)

              Attachments

                Activity

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    neerjakhattar Neerja Khattar
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    1 Start watching this issue

                    Dates

                    • Created:
                      Updated: