- About DM
- What is DM?
- Basic Features
- Advanced Features
- Merge and Migrate Data from Sharded Tables
- Migrate from MySQL Databases that Use GH-ost/PT-osc
- Filter Certain Row Changes Using SQL Expressions
- DM Architecture
- Benchmarks
- Quick Start
- Deploy
- Maintain
- Tools
- Cluster Upgrade
- Manage Data Source
- Manage a Data Migration Task
- Manually Handle Sharding DDL Lock
- Manage Schemas of Tables to be Migrated
- Handle Alerts
- Daily Check
- Usage Scenarios
- Troubleshoot
- Performance Tuning
- Reference
- Secure
- FAQ
- Glossary
- Release Notes
DM 1.0-alpha Benchmark Report
This DM benchmark report describes the test purpose, environment, scenario, and result.
Test purpose
The purpose of this test is to test the performance of DM incremental replication.
Note
The results of the testing might vary based on different environmental dependencies.
Test environment
Machine information
System information:
Machine IP | Operation system | Kernel version | File system type |
---|---|---|---|
192.168.0.6 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
192.168.0.7 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
192.168.0.8 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
192.168.0.9 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
192.168.0.10 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
192.168.0.11 | CentOS Linux release 7.6.1810 | 3.10.0-957.1.3.el7.x86_64 | ext4 |
Hardware information:
Type | 192.168.0.9, 192.168.0.10, 192.168.0.11 | 192.168.0.6, 192.168.0.7, 192.168.0.8 |
---|---|---|
CPU | 8 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz | 4 vCPUs, Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz |
Memory | 16G | 8G |
Disk | 1T Aliyun ESSD | 256G Aliyun ESSD |
Network card | 1 Gigabit Ethernet, 1000Mb/s | 1 Gigabit Ethernet, 1000Mb/s |
Cluster topology
Machine IP | Deployment instance |
---|---|
192.168.0.9 | TiKV * 1, TiDB * 1 |
192.168.0.10 | TiKV * 1 |
192.168.0.11 | TiKV * 1 |
192.168.0.6 | PD * 1, MySQL * 1, DM-worker * 1 |
192.168.0.8 | PD * 1, MySQL * 1, DM-worker * 1 |
192.168.0.7 | PD * 1, DM-master * 1 |
Version information
MySQL version: 5.7.25-log
TiDB version: v3.0.0-beta-27-g6398788
DM version: v1.0.0-alpha-10-g4d01d79
Sysbench version: 1.0.9
Test scenario
Data flow
MySQL1 (192.168.0.8) -> DM-worker1 (192.168.0.6) -> TiDB (192.168.0.9)
Test procedure
- Set up environment
- Use sysbench to create the table and generate the initial data in upstream MySQL
- Start DM-task in the
all
mode - Use sysbench to generate incremental data in upstream MySQL
Use sysbench to generate data load in upstream MySQL
Upstream sysbench test script:
sysbench --test=oltp_insert --tables=2 --num-threads=1024 --mysql-host=192.168.0.8 --mysql-port=3306 --mysql-user=root --mysql-db=dm_poc --db-driver=mysql --report-interval=10 --time=900 run
The structure of the table used for the test:
CREATE TABLE `sbtest` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`k` int(11) NOT NULL DEFAULT '0',
`c` char(120) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
`pad` char(60) CHARSET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `k_1` (`k`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
The deployment and configuration details
// TiKV configuration
sync-log = false
[defaultcf]
block-cache-size = "4GB"
[writecf]
block-cache-size = "4GB"
[raftdb.defaultcf]
block-cache-size = "4GB"
// The task configuration of the sync processing unit in DM
syncers:
worker-count: 256
batch: 100
max-retry: 20
Test result
items | threads | qps | tps | 95% Latency (ms) |
---|---|---|---|---|
MySQL | 1024 | 15.10k | 15.10k | 121.08 |
DM | 256 | 13.89k | 13.89k | 210 (txn execution latency) |
TiDB | - | 18.53k (Begin/Commit 2.4k Replace 13.80k) | 2.27k | 29 |
Monitor screenshots
DM key indicator monitor
TiDB key indicator monitor
What’s on this page
Was this page helpful?