Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: CDH 5.3.0
-
Fix Version/s: None
-
Component/s: HBase
-
Labels:
-
Environment:CentOS7-64
Description
I am working on HBase perf now and suffering from an odd problem: huge perf degradation in hdfs write with CDH5 deployment compared to Apache 1.2.1. Despite the same configurations of Hbase and hdfs… obvious disk imbalance is found with CDH5 deployment as below. My dfs.data.dir uses 8 disks - /mnt/sd
{g..n}1
///////////////////////////CDH5 - Hbase 0.98.6 + Hadoop 2.5///////////////////////
[root@perf2-worker-5 z-cdh5]# du -hs /mnt/sd*
0 /mnt/sdb1
336K /mnt/sdc1
32K /mnt/sdd1
1.1M /mnt/sde1
40K /mnt/sdf1
104K /mnt/sdg1
96K /mnt/sdh1
96K /mnt/sdi1
96K /mnt/sdj1
96K /mnt/sdk1
88K /mnt/sdl1
88K /mnt/sdm1
108K /mnt/sdn1
After execute hbase org.apache.hadoop.hbase.PerformanceEvaluation --presplit=8 --nomapred sequentialWrite 10
[root@perf2-worker-5 z-cdh5]# du -hs /mnt/sd*
0 /mnt/sdb1
712K /mnt/sdc1
32K /mnt/sdd1
1.1M /mnt/sde1
40K /mnt/sdf1
1.3G /mnt/sdg1
2.1G /mnt/sdh1
1.4G /mnt/sdi1
2.0G /mnt/sdj1
1.3G /mnt/sdk1
12G /mnt/sdl1
1.5G /mnt/sdm1
1.2G /mnt/sdn1
///////////////////////////Hbase 0.98.4 + Apache Hadoop 1.2.1///////////////////////
[root@hbase-node1 z]# du -hs /mnt/sd*
232K /mnt/sdc1
0 /mnt/sdcg
36K /mnt/sdd1
1.1M /mnt/sde1
48K /mnt/sdf1
80K /mnt/sdg1
72K /mnt/sdh1
72K /mnt/sdi1
72K /mnt/sdj1
72K /mnt/sdk1
72K /mnt/sdl1
72K /mnt/sdm1
84K /mnt/sdn1
After execute hbase org.apache.hadoop.hbase.PerformanceEvaluation --presplit=8 --nomapred sequentialWrite 10
[root@hbase-node1 z]# du -hs /mnt/sd*
1.5M /mnt/sdc1
0 /mnt/sdcg
44K /mnt/sdd1
1.1M /mnt/sde1
48K /mnt/sdf1
3.6G /mnt/sdg1
3.8G /mnt/sdh1
3.6G /mnt/sdi1
3.6G /mnt/sdj1
3.6G /mnt/sdk1
3.9G /mnt/sdl1
3.3G /mnt/sdm1
4.0G /mnt/sdn1