Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-511

[lily] Error in ReplicationSourceManager - The queue we wanted to close is missing peer-state

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: search-0.9.3
    • Fix Version/s: None
    • Component/s: Search
    • Labels:
      None

      Description

      Errors are in HBase RS logs (can be found by event search from indexer service page) while setup/test hbase indexer. Start this Jira with Search team, please reassign if needed.
      CRITICAL The queue we wanted to close is missing peer-state
      Service: HBASE-1 Role: regionserver (czhucentos-3) Hosts: czhucentos-3.ent.cloudera.com Category: LOG_MESSAGE Collapse
      CATEGORY
      LOG_MESSAGE
      CLUSTER
      Cluster 1 - CDH4
      CLUSTER_ID
      1
      EVENTCODE
      EV_LOG_EVENT
      HOSTS
      czhucentos-3.ent.cloudera.com
      LOG_LEVEL
      ERROR
      ROLE
      regionserver (czhucentos-3)
      ROLE_TYPE
      REGIONSERVER
      SERVICE
      HBASE-1
      SERVICE_TYPE
      HBASE
      SEVERITY
      CRITICAL
      URL
      http://czhucentos-1.ent.cloudera.com:7180/cmf/eventRedirect/01f0dc55-0bd5-498f-86e4-2002fb94102b
      __persist_timestamp
      1375884544088
      __uuid
      01f0dc55-0bd5-498f-86e4-2002fb94102b
      =============
      RS log:
      7:08:40.587 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Choosing peer czhucentos-6.ent.cloudera.com,42644,1375884520564
      7:08:40.587 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Choosing peer czhucentos-4.ent.cloudera.com,57621,1375884520495
      7:08:40.588 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Choosing peer czhucentos-8.ent.cloudera.com,47527,1375884520128
      7:08:40.588 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Choosing peer czhucentos-2.ent.cloudera.com,60482,1375884520470
      7:08:40.598 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Replicating c9f362b9-33aa-4b15-a839-28253696531c -> 298854c6-afdb-34a6-aac1-86b652a3084f
      7:08:40.632 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:08:40.657 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:08:40.705 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:08:40.830 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:09:03.967 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:09:03.986 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Peer regionserver children changed at /ngdata/sep/hbase-slave/Indexer_hbaseIndexer/rs
      7:09:04.087 AM INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager
      Closing the following queue peer-state, currently have 1 and another 0 that were recovered
      7:09:04.088 AM INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager
      Number of deleted recovered sources for peer-state: 0
      7:09:04.088 AM ERROR org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager
      The queue we wanted to close is missing peer-state
      7:09:04.107 AM INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager
      Closing the following queue Indexer_hbaseIndexer, currently have 1 and another 0 that were recovered
      7:09:04.107 AM INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager
      Number of deleted recovered sources for Indexer_hbaseIndexer: 0
      7:09:04.107 AM INFO com.ngdata.sep.impl.fork.ForkedReplicationSource
      Closing source Indexer_hbaseIndexer because: Replication stream was removed by a user
      7:09:04.385 AM INFO org.apache.zookeeper.ZooKeeper
      Session: 0x14058ed3f53008c closed
      7:09:04.386 AM INFO org.apache.zookeeper.ClientCnxn
      EventThread shut down
      7:19:26.422 AM INFO org.apache.hadoop.io.compress.CodecPool
      Got brand-new compressor [.deflate]

        Attachments

          Activity

            People

            • Assignee:
              greid Gabriel Reid
              Reporter:
              czhu Chunshi Zhu
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: