Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: CDH4.3.0
-
Fix Version/s: None
-
Component/s: Hive
-
Environment:[root@backup01.dal05 ~]# uname -a
Linux backup01.dal05.swiftype.net 2.6.32-358.11.1.el6.x86_64 #1 SMP Wed Jun 12 03:34:52 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@backup01.dal05 ~]# java -version
java version "1.7.0_15"
Java(TM) SE Runtime Environment (build 1.7.0_15-b03)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
[ root@backup01.dal05 ~]# uname -a Linux backup01.dal05.swiftype.net 2.6.32-358.11.1.el6.x86_64 #1 SMP Wed Jun 12 03:34:52 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [ root@backup01.dal05 ~]# java -version java version "1.7.0_15" Java(TM) SE Runtime Environment (build 1.7.0_15-b03) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
Description
We at Swiftype are trying to use hiveserver2 from CDH 4.3.0 in production and we've noticed that every time we execute a query on it, hive leaks one zookeeper connection (I see it staying open forever in lsof). I could easily reproduce it by running a simple connect, execute "show tables" query, disconnect nagios check.
Using "SET hive.support.concurrency=false" before executing a query prevents ZK connections from leaking, but I'm not sure how safe it is because Cloudera docs here (http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.3.0/CDH4-Installation-Guide/cdh4ig_topic_18_5.html) say "Failure to do this will prevent HiveServer2 from handling concurrent query requests and may result in data corruption". So, we use hive.support.concurrency=false in our nagios check, but production workload still uses concurrency and this brings our hive server down few times a day because zookeeper servers start rejecting connections from the hive box.