DM-worker Advanced Configuration File

This document details the advanced configuration of DM-worker.

Configuration file template

# Worker Configuration.

# Log configuration.
log-level = "info"
log-file = "dm-worker.log"

# DM-worker listening address.
worker-addr = ":8262"

# Represents a MySQL/MariaDB instance or a replication group.
source-id = "mysql-replica-01"

# Server ID of secondary library for binlog replication.
# Each instance (primary and secondary) in the replication group should have a different server ID.
server-id = 101

# flavor: mysql/mariadb
flavor = "mysql"

# The directory used to store relay log.
relay-dir = "./relay_log"

# Enables gtid in the relay log unit.
enable-gtid = false

relay-binlog-name = ""
relay-binlog-gtid = ""

[from]
host = "127.0.0.1"
user = "root"
password = "Up8156jArvIPymkVC+5LxkAT6rek"
port = 3306

# Relay log purge strategy.
[purge]
interval = 3600
expires = 24
remain-space = 15

# Task status checker.
[checker]
check-enable = true
backoff-rollback = "5m"
backoff-max = "5m"

Configuration parameters

Global

ParameterDescriptionDefault value
log-levelControls the log level: "debug", "info", "warn", "error" or "fatal". For troubleshooting purposes, set it to "debug"."info"
log-fileSpecifies the log file. If not specified, the logs are printed onto the standard output.
worker-addrSpecifies the address of DM-worker which provides services. You can omit the IP address and specify the port number only, such as ":8262".
source-idUniquely identifies a MySQL or MariaDB instance, or a replication group
server-idIdentifies the server ID of DM-worker as a MySQL or MariaDB secondary library. In a replication group, each instance (primary and secondary included) must have a unique server ID. In v1.0.2 and later versions, the server_id is automatically generated by DM.
flavorIndicates the release type of MySQL: "Percona", "mysql" or "mariadb". In v1.0.2 and later versions, DM automatically detects and fills in the release type."mysql"
relay-dirSpecifies the relay log directory."./relay_log"
enable-gtidDetermines whether DM-worker uses GTID to pull the binlog. If the upstream database has enabled the GTID mode and switching the DM-worker connection to another MySQL instance is needed, set it to true.false
relay-binlog-nameSpecifies the file name from which DM-worker starts to pull the binlog. For example, "mysql-bin.000002". It only works when enable_gtid is false. If this parameter is not specified, DM-worker defaults to pulling the binlogs starting from the earliest one. But in v1.0.2 and later versions, DM-worker defaults to pulling the binlogs starting from the latest one.
relay-binlog-gtidSpecifies the GTID from which DM-worker starts to pull the binlog. For example, "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849". It only works when enable_gtid is true. If this parameter is not specified, DM-worker defaults to pulling the binlogs starting from the earliest GTID. But in v1.0.2 and later versions, DM-worker defaults to pulling the binlogs starting from the latest GTID.

[from]

The [from] section contains parameters that affect the connection to the upstream database.

ParameterDescription
hostThe host name of the upstream database.
portThe port number of the upstream database.
userThe username used to connect to the database.
passwordThe password used to connect to the database. Note: Use the password encrypted by dmctl.

[purge]

The [purge] section contains parameters that affect the purge strategy of relay log.

Generally, there is no need to manually configure these parameters unless there is a large amount of relay logs and disk capacity is insufficient.

ParameterDescriptionDefault value
intervalSets the time interval at which relay logs are regularly checked for expiration, in seconds.3600
expiresSets the expiration time for relay logs, in hours. The relay log that is not written by the relay processing unit, or does not need to be read by the existing data migration task will be deleted by DM if it exceeds the expiration time. If this parameter is not specified, the automatic purge is not performed.0
remain-spaceSets the minimum amount of free disk space, in gigabytes. When the available disk space is smaller than this value, DM-worker tries to delete relay logs.15

Note:

DM does not perform automatic purge when either of the following is true:

  • interval is set to 0
  • Both expires and remain-space are set to 0

[checker]

The [checker] section contains parameters that affect the task status checker.

ParameterDescriptionDefault value
check-enableDetermines whether to enable task status checker. If it is set to "true", DM tries to resume data migration task that is suspended due to errors.true
backoff-rollbackSets the time interval for adjusting the waiting time of the automatic recovery."5m0s"
backoff-maxSets the longest time interval for the automatic recovery after errors are detected."5m0s"

Note:

Generally, you only need to determine whether to enable the task status checker through the check-enable parameter. It is not recommended to change the default values of backoff-rollback and backoff-max unless you have an in-depth understanding of these two parameters.