Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-141

Hive MySQL upgrade script from CDH3B3 to CDH3B4 cause performance issue

          Details

          • Type: Bug
          • Status: Resolved
          • Priority: Minor
          • Resolution: Won't Fix
          • Affects Version/s: CDH3b4
          • Fix Version/s: None
          • Component/s: Hive
          • Environment:
            CentOS 5.2, 10 node cluster, hive on a separate client machine.

            Description

            After running the upgrade scripts under /var/lib/hive/scripts/metastore/upgrade/mysql/ against our production MySQL database containing our hive metastore, we noticed a significant slowdown between job submission to the cluster, and actual job execution (even simply appearing on the JobTracker wasn't immediate).

            We tried several different things, reviewing DNS issues, modifying /etc/hosts, reviewing datanode, tasktracker, jobtracker, and namenode log files in DEBUG mode, etc.

            In the end, removing hive database from MySQL, and letting hive recreate the schema from scratch resolved the issue.

              Attachments

                Activity

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    stunney Stephen Tunney
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: