- Table Routing
- Block and Allow Lists
- Binlog Event Filter
- Migration Delay Monitoring
- Usage Scenarios
- Manage the DM Cluster
- Manage Migration Tasks
- Migrate from MySQL-compatible Database
- DM Portal
Data Migration (DM) provides a variety of features, including table routing, block & allow table lists, and binlog event filter. However, these features also increase the complexity of using DM, especially when users are modifying DM task configurations.
To address this problem, DM provides a simple web application, DM Portal. DM Portal enables users to visually configure the required migration tasks, and generates a
task.yaml file that can be directly executed by DM.
This sections describes the features of DM Portal.
DM Portal supports three migration modes:
- Full migration
- Incremental replication
- All (full + incremental)
DM Portal supports configuring table routing, and merging sharded schemas and tables in DM.
DM Portal supports configuring the binlog event filter in schemas and tables.
DM Portal supports generating configuration files and downloading these files to your local computer. Meanwhile, it automatically creates a file in the
/tmp/ directory on the dm-portal server.
Currently, DM Portal's visualized pages cover most DM configuration scenarios, but with the following restrictions:
The SQL pattern of binlog event filter is not supported.
The editing feature does not support parsing the
task.yamlfile created by the user. The user can only edit the
task.yamlfile generated by the page.
The editing feature does not support modifying the instance configuration. If the user need to adjust the instance configuration, the
task.yamlfile has to be regenerated.
The upstream instance configuration on the page can only be used to obtain the upstream table schema. The related upstream instance information still needs to be configured in DM-worker.
In the generated
task.yamlfile, mydumper-path is
./bin/mydumperby default. If you use another path, modify the generated
This section describes how to deploy DM Portal in two ways: using binary or DM Ansible.
Download DM Portal at dm-portal-latest-linux-amd64.tar.gz. To start DM Portal, run the
- If you run DM Portal locally, visit
127.0.0.1:8280in your browser.
- If you run DM Portal on a server, configure a proxy on the server.
To deploy DM Portal using DM Ansible, refer to Deploy Data Migration Using DM-Ansible for details.
This section describes how to use DM Portal.
This feature is used to create a
Access the dm-portal page, and click Create New Rule.
This feature is used to fill in the task name and select a task type.
Create New Sync Rule is already selected.
- Fill in the task name.
- Choose a task type.
This feature is used to configure the upstream and downstream instance information, including Host, Port, Username, and Password.
Task Name and Task Type are already filled in.
If you choose Incremental or All in Task Type, you need to configure binlog-file and binlog-pos when configuring the upstream instance information.
- Fill in the upstream instance information.
- Fill in the downstream instance information.
- Click Next.
This feature is used to filter the upstream binlog. You can choose the DDL or DML that needs to be filtered. The filter configured on the database is automatically inherited by tables in that database.
- The upstream and downstream instance information is configured.
- The connection is verified.
- The binlog filter configuration can only be modified in the upstream instance. Once the database or table is moved to the downstream instance, the configuration cannot be modified.
- The binlog filter configured on the database is automatically inherited by tables in that database.
Select the databases or tables that need to be configured.
Click the Edit button, and select the binlog types to be filtered.
This feature is used to perform the following operations:
- Select the databases and tables that need to be synced, modify their names, and merge databases and tables
- Revert the last operation
- Reset all configurations of table routing
After the task configuration is completed, DM Portal generates the corresponding
The required binlog filter rules are configured.
- Batch operation is not supported when you merge databases and tables. You can only drag them one by one.
- You can only drag tables when you merge databases and tables. You cannot drag databases.
Select the databases and tables that need to be synced from Upstream Instance.
Click the Move button and move the selected databases and tables to Downstream Instance.
Right click the databases and tables to rename them.
Select the required table to perform the following operation:
To merge two tables, drag the table onto another table
To move the table to an existing database, drag the table onto the database
To move the table to a new database, drag the table onto the
Click Go Back to undo the last operation.
Click Reset to clear the downstream instance.
Click Finish & Download. DM Portal automatically downloads the
task.yamlfile to the local computer, and creates a
task.yamlconfiguration file in the
/tmp/directory on the DM Portal server.
- DM Portal Overview