- About DM
- What is DM?
- Basic Features
- Advanced Features
- Merge and Migrate Data from Sharded Tables
- Migrate from MySQL Databases that Use GH-ost/PT-osc
- Filter Certain Row Changes Using SQL Expressions
- DM Architecture
- Quick Start
- Cluster Upgrade
- Manage Data Source
- Manage a Data Migration Task
- Manually Handle Sharding DDL Lock
- Manage Schemas of Tables to be Migrated
- Handle Alerts
- Daily Check
- Usage Scenarios
- Performance Tuning
- Release Notes
This document introduces the main usage scenarios of TiDB Data Migration (DM) and the usage suggestions.
If you want to use TiDB as a secondary database of the upstream MySQL, Amazon Aurora, or MariaDB database (that is, import the full data of the upstream instance to TiDB, and replicate the incremental data to TiDB in real-time once data changes are made), you can configure the data migration task according to the following rules:
target-databasebased on the connection information of the downstream TiDB.
mysql-instancesbased on the ID information of the upstream MySQL/MariaDB instances.
- Use the default setting of concurrency control parameters or configure
- No need to configure
If there are multiple application data in MySQL/MariaDB, but only part of the data needs to be migrated to TiDB, you can configure the migration task by referring to Use TiDB as a secondary database of MySQL/MariaDB. Then configure
block-allow-list as needed.
To migrate upstream data to a schema or table with a different name in the downstream database, you can configure
For some archiving scenarios, some data may be periodically cleaned up by executing the
DROP TABLE command in the upstream or through other means. To make sure all data are retained in the downstream TiDB, you can disable these data clearing operations by configuring
For more information, refer to Data Migration Simple Usage Scenario.
If there are multiple sharded tables in multiple sharded schemas in the upstream MySQL/MariaDB, to merge them into one table or schema when migrating to TiDB, you can rename the table or the schema in the upstream database by configuring
route-rules, and then these tables can be merged into the same downstream schema or table. For details, refer to Data Migration Shard Merge Scenario and Best Practices of Data Migration in the Shard Merge Scenario.
Specifically, DM supports the migration of DDL. For details, refer to Merge and Migrate Data from Sharded Tables.
If you only need to migrate some application data or filter out some operations, refer to Migrate some applicationb data from MySQL/MariaDB.
In MySQL ecosystem, tools such as pt-osc or gh-ost are often used to perform online DDL operations. DM also supports such scenarios. You can enable support for pt-osc or gh-ost by configuring the
online-ddl parameter. For details, refer to DM Online DDL Scheme.
When you use MySQL/MariaDB, a primary-secondary cluster can be built to improve the read performance and to ensure data safety. If the upstream MySQL/MariaDB instance connected by DM-worker is unavailable for some reason when you use DM to migrate data, you need to switch the DM-worker connection to another MySQL/MariaDB instance in the upstream primary-secondary cluster. For such scenarios, DM provides good support, but there are some limitations. For details, see Switch DM-worker connection between upstream MySQL instances.