Doc Menu

TiDB Sysbench Performance Test Report -- v4.0 vs. v3.0

Test purpose

This test aims to compare the Sysbench performance of TiDB 4.0 and TiDB 3.0 in the Online Transactional Processing (OLTP) scenario.

Test environment (AWS EC2)

Hardware configuration

Service typeEC2 typeInstance count
PDm5.xlarge3
TiKVi3.4xlarge3
TiDBc5.4xlarge3
Sysbenchm5.4xlarge1

Software version

Service typeSoftware version
PD3.0 and 4.0
TiDB3.0 and 4.0
TiKV3.0 and 4.0
Sysbench1.0.20

Parameter configuration

TiDB v3.0 configuration

log.level: "error"
performance.max-procs: 20
prepared-plan-cache.enabled: true
tikv-client.max-batch-wait-time: 2000000

TiKV v3.0 configuration

storage.scheduler-worker-pool-size: 5
raftstore.store-pool-size: 3
raftstore.apply-pool-size: 3
rocksdb.max-background-jobs: 3
raftdb.max-background-jobs: 3
raftdb.allow-concurrent-memtable-write: true
server.grpc-concurrency: 6
readpool.storage.normal-concurrency: 10
readpool.coprocessor.normal-concurrency: 5

TiDB v4.0 configuration

log.level: "error"
performance.max-procs: 20
prepared-plan-cache.enabled: true
tikv-client.max-batch-wait-time: 2000000

TiKV v4.0 configuration

storage.scheduler-worker-pool-size: 5
raftstore.store-pool-size: 3
raftstore.apply-pool-size: 3
rocksdb.max-background-jobs: 3
raftdb.max-background-jobs: 3
raftdb.allow-concurrent-memtable-write: true
server.request-batch-enable-cross-command: false
server.grpc-concurrency: 6
readpool.unifiy-read-pool: true
readpool.unified.min-thread-count: 5
readpool.unified.max-thread-count: 20
readpool.storage.normal-concurrency: 10
pessimistic-txn.pipelined: true

Global variable configuration

set global tidb_hashagg_final_concurrency=1;
set global tidb_hashagg_partial_concurrency=1;
set global tidb_disable_txn_auto_retry=0;

Test plan

  1. Deploy TiDB v4.0 and v3.0 using TiUP.
  2. Use Sysbench to import 16 tables, each table with 10 million rows of data.
  3. Execute the analyze table statement on each table.
  4. Back up the data used for restore before different concurrency tests, which ensures data consistency for each test.
  5. Start the Sysbench client to perform the point_select, read_write, update_index, and update_non_index tests. Perform stress tests on TiDB via AWS NLB. In each type of test, the warm-up takes 1 minute and the test takes 5 minutes.
  6. After each type of test is completed, stop the cluster, overwrite the cluster with the backup data in step 4, and restart the cluster.

Prepare test data

Execute the following command to prepare the test data:

sysbench oltp_common \
    --threads=16 \
    --rand-type=uniform \
    --db-driver=mysql \
    --mysql-db=sbtest \
    --mysql-host=$aws_nlb_host \
    --mysql-port=$aws_nlb_port \
    --mysql-user=root \
    --mysql-password=password \
    prepare --tables=16 --table-size=10000000

Perform the test

Execute the following command to perform the test.

sysbench $testname \
    --threads=$threads \
    --time=300 \
    --report-interval=1 \
    --rand-type=uniform \
    --db-driver=mysql \
    --mysql-db=sbtest \
    --mysql-host=$aws_nlb_host \
    --mysql-port=$aws_nlb_port \
    run --tables=16 --table-size=10000000

Test results

Point Select performance

Threadsv3.0 QPSv3.0 95% latency (ms)v4.0 QPSv4.0 95% latency (ms)QPS improvement
150117085.7011.667118165.13571.6080.92%
300200621.44712.615207774.08592.0323.57%
600283928.93234.569320673.3423.30412.94%
900343218.26246.686383913.38554.65211.86%
1200347200.23668.092408929.43726.31817.78%
1500366406.276710.562418268.88567.98514.15%

Compared with v3.0, the Point Select performance of TiDB v4.0 has increased by 14%.

Point Select

Update Non-index performance

Threadsv3.0 QPSv3.0 95% latency (ms)v4.0 QPSv4.0 95% latency (ms)QPS improvement
15015446.4102411.44616954.3997110.8449.76%
30022276.1557217.31924364.4468916.7069.37%
60028784.8835329.19431635.7083328.1629.90%
90032194.1554842.61135787.6607838.94211.16%
120033954.6911458.92338552.6315851.01813.54%
150035412.003274.46440859.6375562.19315.38%

Compared with v3.0, the Update Non-index performance of TiDB v4.0 has increased by 15%.

Update Non-index

Update Index performance

Threadsv3.0 QPSv3.0 95% latency (ms)v4.0 QPSv4.0 95% latency (ms)QPS improvement
15011164.4057116.70611954.7363516.4087.08%
30014460.9805728.16215243.4089928.1625.41%
60017112.7303653.8518535.0751550.1078.31%
90018233.8342686.00220339.690170.54811.55%
120018622.50283127.80521390.2512294.10414.86%
150018980.34447170.47922359.996114.71717.81%

Compared with v3.0, the Update Index performance of TiDB v4.0 has increased by 17%.

Update Index

Read-write performance

Threadsv3.0 QPSv3.0 95% latency (ms)v4.0 QPSv4.0 95% latency (ms)QPS improvement
15043768.3363371.8353912.6370559.99323.18%
30055655.63589121.08571327.2133697.55528.16%
60064642.96992223.34484487.75483176.73130.70%
90068947.25293325.98490177.94612257.9530.79%
120071334.80099434.82992779.71507344.07830.06%
150072069.9115580.01795088.50812434.82931.94%

Compared with v3.0, the read-write performance of TiDB v4.0 has increased by 31%.

Read Write