DM 1.0-alpha Benchmark Report
This DM benchmark report describes the test purpose, environment, scenario, and result.
Test purpose
The purpose of this test is to test the performance of DM incremental replication.
Test environment
Machine information
System information:
| Machine IP | Operation system | Kernel version | File system type |
|---|---|---|---|
| 192.168.0.6 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
| 192.168.0.7 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
| 192.168.0.8 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
| 192.168.0.9 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
| 192.168.0.10 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
| 192.168.0.11 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
Hardware information:
| Type | 192.168.0.9, 192.168.0.10, 192.168.0.11 | 192.168.0.6, 192.168.0.7, 192.168.0.8 |
|---|---|---|
| CPU | 8 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz | 4 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz |
| Memory | 16G | 8G |
| Disk | 1T Aliyun ESSD | 256G Aliyun ESSD |
| Network card | 1 Gigabit Ethernet, 1000Mb/s | 1 Gigabit Ethernet, 1000Mb/s |
Cluster topology
| Machine IP | Deployment instance |
|---|---|
| 192.168.0.9 | TiKV * 1, TiDB * 1 |
| 192.168.0.10 | TiKV * 1 |
| 192.168.0.11 | TiKV * 1 |
| 192.168.0.6 | PD * 1, MySQL * 1, DM-worker * 1 |
| 192.168.0.8 | PD * 1, MySQL * 1, DM-worker * 1 |
| 192.168.0.7 | PD * 1, DM-master * 1 |
Version information
MySQL version: 5.7.25-log
TiDB version: v3.0.0-beta-27-g6398788
DM version: v1.0.0-alpha-10-g4d01d79
Sysbench version: 1.0.9
Test scenario
Data flow
MySQL1 (192.168.0.8) -> DM-worker1 (192.168.0.6) -> TiDB (192.168.0.9)
Test procedure
- Set up environment
- Use sysbench to create the table and generate the initial data in upstream MySQL
- Start DM-task in the
allmode - Use sysbench to generate incremental data in upstream MySQL
Use sysbench to generate data load in upstream MySQL
Upstream sysbench test script:
sysbench --test=oltp_insert --tables=2 --num-threads=1024 --mysql-host=192.168.0.8 --mysql-port=3306 --mysql-user=root --mysql-db=dm_poc --db-driver=mysql --report-interval=10 --time=900 run
The structure of the table used for the test:
CREATE TABLE `sbtest` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`k` int(11) NOT NULL DEFAULT '0',
`c` char(120) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
`pad` char(60) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `k_1` (`k`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
The deployment and configuration details
// TiKV configuration
sync-log = false
[defaultcf]
block-cache-size = "4GB"
[writecf]
block-cache-size = "4GB"
[raftdb.defaultcf]
block-cache-size = "4GB"
// The task configuration of the sync processing unit in DM
syncers:
worker-count: 256
batch: 100
max-retry: 20
Test result
| items | threads | qps | tps | 95% Latency (ms) |
|---|---|---|---|---|
| MySQL | 1024 | 15.10k | 15.10k | 121.08 |
| DM | 256 | 13.89k | 13.89k | 210 (txn execution latency) |
| TiDB | - | 18.53k (Begin/Commit 2.4k Replace 13.80k) | 2.27k | 29 |
Monitor screenshots
DM key indicator monitor
TiDB key indicator monitor
