使用 ycsb 工具对 hyperbase 表进行压测的方法说明

  性能相关
内容纲要

概要描述

提供一种使用 YCSB 进行 hbase 表性能压的方法：

环境配置：虚拟机 4节点（18core+58G），hyperbase6.2.2 （regionserver 3，20G 内存）

详细说明

  1. 建表
  2. 关闭WAL
  3. 随机写测试
  4. 随机读测试

建表

create 'usertable40', {NAME => 'family', COMPRESSION => 'SNAPPY'}, {SPLITS =>(1..39).map {|i| "user#{1000+i*(9999-1000)/39}"}}

关闭WAL

生产环境下为了避免数据丢失,一般选择打开WAL.由于benchmark测试的是极限性能,故这里关闭WAL.

alter 'usertable40', DURABILITY => 'SKIP_WAL'

[root@kv1~/sf/ycsb/ycsb-hbase14-binding-0.17.0]# hbase shell
2024-04-02 10:00:56,741 INFO util.KerberosUtil: Using principal pattern: HTTP/_HOST
HBase Shell; enter 'help' for list of supported commands.
Type "exit" to leave the HBase Shell
Version 1.3.1-transwarp-8.1.0, r5b035cf28516e9a9363d7fac9cc7c1a07e8a3430, Tue Nov 16 17:33:51 CST 2021

hbase(main):001:0> create 'usertable40', {NAME => 'family', COMPRESSION => 'SNAPPY'}, {SPLITS =>(1..39).map {|i| "user#{1000+i*(9999-1000)/39}"}}
0 row(s) in 5.1220 seconds

=> Hbase::Table - usertable40
hbase(main):002:0> alter 'usertable40', DURABILITY => 'SKIP_WAL'
Updating all regions with the new schema...
6/40 regions updated.
15/40 regions updated.
27/40 regions updated.
38/40 regions updated.
40/40 regions updated.
Done.
0 row(s) in 7.3730 seconds

[100%随机写]

bin/ycsb load hbase14 -P workloads/`ycsb_insert100 `-cp /etc/hyperbase1/conf:/etc/hdfs1/conf -p table=usertable40 -threads 40 -p columnfamily=family -p recordcount=10000000 -s

ycsb_insert100 配置文件：

# Yahoo! Cloud System Benchmark
# Workload A: Update heavy workload
#   Application example: Session store recording recent actions
#                        
#   Read/update ratio: 50/50
#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)
#   Request distribution: zipfian

workload=site.ycsb.workloads.CoreWorkload

readproportion=0
updateproportion=0
scanproportion=0
insertproportion=1

clientbuffering=true
writebuffersize=4194304
  • readproportion=0：指定读取操作的比例为0%。
  • updateproportion=0：指定更新操作的比例为0%。
  • scanproportion=0：指定扫描操作的比例为0%。
  • insertproportion=1：指定插入操作的比例为100%。
  • clientbuffering表示客户端缓存
  • writebuffersize表示缓存大小,这里设置4M即可。
  • workload=site.ycsb.workloads.CoreWorkload：指定使用的工作负载类型为CoreWorkload，这是YCSB提供的一种默认工作负载类型。
[root@kv1~/sf/ycsb/ycsb-hbase14-binding-0.17.0]# bin/ycsb load hbase14 -P workloads/ycsb_insert100 -cp /etc/hyperbase1/conf:/etc/hdfs1/conf -p table=usertable40 -threads 40 -p columnfamily=family -p recordcount=10000000 -s
/usr/lib/jvm/java-1.8.0/bin/java -cp /etc/hyperbase1/conf:/etc/hdfs1/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/hbase14-binding/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/core-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core4-4.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-mapper-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-core-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/HdrHistogram-2.1.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase10-binding-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase-shaded-client-1.4.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/commons-logging-1.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core-3.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/log4j-1.2.17.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-log4j12-1.7.7.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-api-1.7.25.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/findbugs-annotations-1.3.9-1.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase14-binding-0.17.0.jar site.ycsb.Client -db site.ycsb.db.hbase14.HBaseClient14 -P workloads/ycsb_insert100 -p table=usertable40 -threads 40 -p columnfamily=family -p recordcount=10000000 -s -load
Command line: -db site.ycsb.db.hbase14.HBaseClient14 -P workloads/ycsb_insert100 -p table=usertable40 -threads 40 -p columnfamily=family -p recordcount=10000000 -s -load
YCSB Client 0.17.0

Loading workload...
Starting test.
2024-04-02 10:20:47:469 0 sec: 0 operations; est completion in 0 second 
2024-04-02 10:20:48,828 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x5e369105 connecting to ZooKeeper ensemble=kv2:2181,kv3:2181,kv4:2181
8,902 DEBUG configuration.ConfigurationUtils: Loading configuration from the context classpath (hadoop-metrics2-hbase.properties)
2024-04-02 10:20:48,914 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties
2024-04-02 10:20:49,040 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2024-04-02 10:20:49,040 INFO impl.MetricsSystemImpl: HBase metrics system started
2024-04-02 10:20:49,076 INFO metrics.MetricRegistries: Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:host.name=kv1
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_181
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64/jre
2024-04-02 10:20:49,100 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/etc/hyperbase1/conf:/etc/hdfs1/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/hbase14-binding/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/core-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core4-4.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-mapper-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-core-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/HdrHistogram-2.1.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase10-binding-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase-shaded-client-1.4.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/commons-logging-1.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core-3.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/log4j-1.2.17.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-log4j12-1.7.7.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-api-1.7.25.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/findbugs-annotations-1.3.9-1.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase14-binding-0.17.0.jar
2024-04-02 10:24:41,247 DEBUG zookeeper.ClientCnxn: An exception was thrown while closing send thread for session 0xff8e83ebd9a49bdc : Unable to read additional data from server sessionid 0xff8e83ebd9a49bdc, likely server has closed socket
2024-04-02 10:24:41,247 DEBUG ipc.AbstractRpcClient: Stopping rpc client
2024-04-02 10:24:41,247 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0xff8e83ebd9a49bdc
2024-04-02 10:24:41:250 233 sec: 10000000 operations; 24821.42 current ops/sec; [CLEANUP: Count=30, Max=4093951, Min=300288, Avg=2253998.93, 90=3203071, 99=4093951, 99.9=4093951, 99.99=4093951] [INSERT: Count=94792, Max=1208319, Min=4, Avg=320.86, 90=9, 99=323, 99.9=64063, 99.99=431615] [UPDATE: Count=30, Max=4093951, Min=228096, Avg=2251249.07, 90=3203071, 99=4093951, 99.9=4093951, 99.99=4093951] 
[OVERALL], RunTime(ms), 233820
[OVERALL], Throughput(ops/sec), 42767.941151312974
[TOTAL_GCS_PS_Scavenge], Count, 44
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 17834
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 7.627234624925156
[TOTAL_GCS_PS_MarkSweep], Count, 3
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 792
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.3387220939183988
[TOTAL_GCs], Count, 47
[TOTAL_GC_TIME], Time(ms), 18626
[TOTAL_GC_TIME_%], Time(%), 7.965956718843555
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2919273.6
[CLEANUP], MinLatency(us), 300288
[CLEANUP], MaxLatency(us), 9805823
[CLEANUP], 95thPercentileLatency(us), 7434239
[CLEANUP], 99thPercentileLatency(us), 9805823
[INSERT], Operations, 10000000
[INSERT], AverageLatency(us), 897.7104096
[INSERT], MinLatency(us), 4
[INSERT], MaxLatency(us), 15458303
[INSERT], 95thPercentileLatency(us), 15
[INSERT], 99thPercentileLatency(us), 94
[INSERT], Return=OK, 10000000
[UPDATE], Operations, 40
[UPDATE], AverageLatency(us), 2917108.8
[UPDATE], MinLatency(us), 228096
[UPDATE], MaxLatency(us), 9805823
[UPDATE], 95thPercentileLatency(us), 7434239
[UPDATE], 99thPercentileLatency(us), 9805823

测试结论

  • 数据量：1000w数据量
  • region数量：40
  • 线程数：40
  • 吞吐量（ops/sec）：42767.941151312974
  • 平均吞吐量（ops/sec）：14255.98

[100%随机读]

./bin/ycsb run hbase14 -P workloads/ycsb_read100 -cp /etc/hyperbase1/conf:/etc/hdfs1/conf -p table=usertable40 -threads 40 -p columnfamily=family -p operationcount=5000000 -s

ycsb_read100配置文件如下：

# Yahoo! Cloud System Benchmark
# Workload A: Update heavy workload
#   Application example: Session store recording recent actions
#                        
#   Read/update ratio: 50/50
#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)
#   Request distribution: zipfian

workload=site.ycsb.workloads.CoreWorkload

readproportion=1
updateproportion=0
scanproportion=0
insertproportion=0
  • readproportion=1：指定读取操作的比例为100%。
  • updateproportion=0：指定更新操作的比例为0%。
  • scanproportion=0：指定扫描操作的比例为0%。
  • insertproportion=0：指定插入操作的比例为0%。
[root@kv1~/sf/ycsb/ycsb-hbase14-binding-0.17.0]# ./bin/ycsb run hbase14 -P workloads/ycsb_read100 -cp /etc/hyperbase1/conf:/etc/hdfs1/conf -p table=usertable40 -threads 40 -p columnfamily=family -p operationcount=5000000 -s
/usr/lib/jvm/java-1.8.0/bin/java -cp /etc/hyperbase1/conf:/etc/hdfs1/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/hbase14-binding/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/conf:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/core-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core4-4.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-mapper-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/jackson-core-asl-1.9.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/HdrHistogram-2.1.4.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase10-binding-0.17.0.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase-shaded-client-1.4.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/commons-logging-1.2.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/htrace-core-3.1.0-incubating.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/log4j-1.2.17.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-log4j12-1.7.7.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/slf4j-api-1.7.25.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/findbugs-annotations-1.3.9-1.jar:/root/sf/ycsb/ycsb-hbase14-binding-0.17.0/lib/hbase14-binding-0.17.0.jar site.ycsb.Client -db site.ycsb.db.hbase14.HBaseClient14 -P workloads/ycsb_read100 -p table=usertable40 -threads 40 -p columnfamily=family -p operationcount=5000000 -s -t
Command line: -db site.ycsb.db.hbase14.HBaseClient14 -P workloads/ycsb_read100 -p table=usertable40 -threads 40 -p columnfamily=family -p operationcount=5000000 -s -t
YCSB Client 0.17.0

Loading workload...
Starting test.
2024-04-02 11:07:32:638 0 sec: 0 operations; est completion in 0 second 
2024-04-02 11:16:21,159 DEBUG ipc.AbstractRpcClient: Stopping rpc client
2024-04-02 11:16:21,159 DEBUG zookeeper.ClientCnxn: An exception was thrown while closing send thread for session 0xff8e83f013219ea8 : Unable to read additional data from server sessionid 0xff8e83f013219ea8, likely server has closed socket
2024-04-02 11:16:21,159 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0xff8e83f013219ea8
2024-04-02 11:16:21:217 528 sec: 5000000 operations; 8403.52 current ops/sec; [READ: Count=325, Max=341247, Min=600, Avg=5422.02, 90=9503, 99=26143, 99.9=341247, 99.99=341247] [CLEANUP: Count=78, Max=53343, Min=2, Avg=704.21, 90=27, 99=440, 99.9=53343, 99.99=53343] [READ-FAILED: Count=72237, Max=345087, Min=404, Avg=3350.37, 90=4111, 99=14687, 99.9=287999, 99.99=343807] 
[OVERALL], RunTime(ms), 528638
[OVERALL], Throughput(ops/sec), 9458.268228920357
[TOTAL_GCS_PS_Scavenge], Count, 30
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 383
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.07245033463352993
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 50
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.009458268228920357
[TOTAL_GCs], Count, 31
[TOTAL_GC_TIME], Time(ms), 433
[TOTAL_GC_TIME_%], Time(%), 0.08190860286245029
[READ], Operations, 23204
[READ], AverageLatency(us), 14017.767195311153
[READ], MinLatency(us), 600
[READ], MaxLatency(us), 1038335
[READ], 95thPercentileLatency(us), 52415
[READ], 99thPercentileLatency(us), 217855
[READ], Return=OK, 23204
[READ], Return=NOT_FOUND, 4976796
[CLEANUP], Operations, 80
[CLEANUP], AverageLatency(us), 697.075
[CLEANUP], MinLatency(us), 2
[CLEANUP], MaxLatency(us), 53343
[CLEANUP], 95thPercentileLatency(us), 48
[CLEANUP], 99thPercentileLatency(us), 823
[READ-FAILED], Operations, 4976796
[READ-FAILED], AverageLatency(us), 4131.60981422586
[READ-FAILED], MinLatency(us), 352
[READ-FAILED], MaxLatency(us), 1155071
[READ-FAILED], 95thPercentileLatency(us), 6307
[READ-FAILED], 99thPercentileLatency(us), 24655

测试结论

  • 数据量：1000w数据量
  • region数量：40
  • 线程数：40
  • 吞吐量（ops/sec）：9458.268228920357
  • 平均吞吐量（ops/sec）：3152.756

注意本测试为虚拟机测试，并不能代表hbase的瓶颈性能，建议使用物理机进行测试

这篇文章对您有帮助吗？

平均评分 0 / 5. 次数: 0

尚无评价，您可以第一个评哦！

非常抱歉，这篇文章对您没有帮助.

烦请您告诉我们您的建议与意见，以便我们改进，谢谢您。