DM 1.0-alpha Benchmark Report

This DM benchmark report describes the test purpose, environment, scenario, and result.

Test purpose

The purpose of this test is to test the performance of DM incremental replication.

Note:

The results of the testing might vary based on different environmental dependencies.

Test environment

Machine information

System information:

Machine IPOperation systemKernel versionFile system type
192.168.0.6CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4
192.168.0.7CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4
192.168.0.8CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4
192.168.0.9CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4
192.168.0.10CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4
192.168.0.11CentOS Linux release 7.6.18103.10.0-957.1.3.el7.x86_64ext4

Hardware information:

Type192.168.0.9, 192.168.0.10, 192.168.0.11192.168.0.6, 192.168.0.7, 192.168.0.8
CPU8 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz4 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz
Memory16G8G
Disk1T Aliyun ESSD256G Aliyun ESSD
Network card1 Gigabit Ethernet, 1000Mb/s1 Gigabit Ethernet, 1000Mb/s

Cluster topology

Machine IPDeployment instance
192.168.0.9TiKV * 1, TiDB * 1
192.168.0.10TiKV * 1
192.168.0.11TiKV * 1
192.168.0.6PD * 1, MySQL * 1, DM-worker * 1
192.168.0.8PD * 1, MySQL * 1, DM-worker * 1
192.168.0.7PD * 1, DM-master * 1

Version information

  • MySQL version: 5.7.25-log

  • TiDB version: v3.0.0-beta-27-g6398788

  • DM version: v1.0.0-alpha-10-g4d01d79

  • Sysbench version: 1.0.9

Test scenario

Data flow

MySQL1 (192.168.0.8) -> DM-worker1 (192.168.0.6) -> TiDB (192.168.0.9)

Test procedure

  • Set up environment
  • Use sysbench to create the table and generate the initial data in upstream MySQL
  • Start DM-task in the all mode
  • Use sysbench to generate incremental data in upstream MySQL

Use sysbench to generate data load in upstream MySQL

Upstream sysbench test script:

sysbench --test=oltp_insert --tables=2 --num-threads=1024 --mysql-host=192.168.0.8 --mysql-port=3306 --mysql-user=root --mysql-db=dm_poc --db-driver=mysql --report-interval=10 --time=900 run

The structure of the table used for the test:

CREATE TABLE `sbtest` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `k` int(11) NOT NULL DEFAULT '0',
  `c` char(120) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
  `pad` char(60) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
  PRIMARY KEY (`id`),
  KEY `k_1` (`k`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin

The deployment and configuration details

// TiKV configuration
sync-log = false
[defaultcf]
block-cache-size = "4GB"
[writecf]
block-cache-size = "4GB"
[raftdb.defaultcf]
block-cache-size = "4GB"

// DM task syncer unit configuration
syncer:
    worker-count: 256
    batch: 100
    max-retry: 20

Test result

itemsthreadsqpstps95% Latency (ms)
MySQL102415.10k15.10k121.08
DM25613.89k13.89k210 (txn execution latency)
TiDB-18.53k (Begin/Commit 2.4k Replace 13.80k)2.27k29

Monitor screenshots

DM key indicator monitor

DM benchmark

TiDB key indicator monitor

TiDB transaction's OPS and duration TiDB's query duration and QPS