- Introduction
- Deploy
- Deploy TiDB Cluster
- Deploy Heterogeneous Cluster
- Deploy TiFlash
- Deploy TiCDC
- Deploy TiDB Binlog
- Deploy TiDB Enterprise Edition
- Deploy Multiple Sets of TiDB Operator
- Deploy Monitoring
- Secure
- Operate
- Scale TiDB Cluster
- Backup and Restore
- Backup and Restore with S3-Compatible Storage
- Backup and Restore with GCS
- Backup and Restore with Persistent Volumes
- Disaster Recovery
- Reference
- Tools
- Configure
- Release Notes
- v1.1
- v1.0
- v0
Deploy Multiple Sets of TiDB Operator
This document describes how to deploy multiple sets of TiDB Operator to manage different TiDB clusters.
Note:
- Currently, you can only deploy multiple
tidb-controller-manager
s andtidb-scheduler
s.- If you have deployed multiple sets of TiDB Operator and only some of them enable Advanced StatefulSet, the same TidbCluster Custom Resource (CR) cannot be switched among these TiDB Operator.
Related parameters
To support deploying multiple sets of TiDB Operator, the following parameters are added to the values.yaml
file in the tidb-operator
chart:
appendReleaseSuffix
If this parameter is set to
true
, when you deploy TiDB Operator, the Helm chart automatically adds a suffix (-{{ .Release.Name }}
) to the name of resources related totidb-controller-manager
andtidb-scheduler
.For example, if you execute
helm install canary pingcap/tidb-operator ...
, the name of thetidb-controller-manager
deployment istidb-controller-manager-canary
.If you need to deploy multiple sets of TiDB Operator, set this parameter to
true
.Default value:
false
.controllerManager.create
Controls whether to create
tidb-controller-manager
.Default value:
true
.controllerManager.selector
Sets the
-selector
parameter fortidb-controller-manager
. The parameter is used to filter the CRs controlled bytidb-controller-manager
according to the CR labels. If multiple selectors exist, the selectors are inand
relationship.Default value:
[]
(tidb-controller-manager
controls all CRs).Example:
selector: - canary-release=v1 - k1==v1 - k2!=v2
scheduler.create
Controls whether to create
tidb-scheduler
.Default value:
true
.
Deploy
Deploy the first TiDB Operator.
Refer to Deploy TiDB Operator to deploy the first TiDB Operator. Add the following configuration in the
values.yaml
:controllerManager: selector: - user=dev
Deploy the TiDB cluster.
Refer to Configure the TiDB Cluster to configure the TidbCluster CR, and configure
labels
to match theselector
set in the last step. For example:apiVersion: pingcap.com/v1alpha1 kind: TidbCluster metadata: name: basic1 labels: user: dev spec: ...
If
labels
is not set when you deploy the TiDB cluster, you can configurelabels
by running the following command:kubectl -n ${namespace} label tidbcluster ${cluster_name} user=dev
Refer to Deploy TiDB in General Kubernetes to deploy the TiDB cluster. Confirm that each component in the cluster is started normally.
Deploy the second TiDB Operator.
Refer to Deploy TiDB Operator to deploy the second TiDB Operator without
tidb-scheduler
. Add the following configuration in thevalues.yaml
:controllerManager: selector: - user=qa appendReleaseSuffix: true scheduler: create: false
Note:
If you configure
scheduler.create: true
, atidb-scheduler
named{{ .scheduler.schedulerName }}-{{.Release.Name}}
is created. To use thistidb-scheduler
, you need to configurespec.schedulerName
in the TidbCluster CR to the name of this scheduler.Deploy the TiDB cluster.
Refer to Configure the TiDB Cluster to configure the TidbCluster CR, and configure
labels
to match theselector
set in the last step. For example:apiVersion: pingcap.com/v1alpha1 kind: TidbCluster metadata: name: basic2 labels: user: qa spec: ...
If
labels
is not set when you deploy the TiDB cluster, you can configurelabels
by running the following command:kubectl -n ${namespace} label tidbcluster ${cluster_name} user=qa
Refer to Deploy TiDB in General Kubernetes to deploy the TiDB cluster. Confirm that each component in the cluster is started normally.
View the logs of the two sets of TiDB Operator, and confirm that each TiDB Operator manages the TiDB cluster that matches the corresponding selectors.
For example:
View the log of
tidb-controller-manager
of the first TiDB Operator:kubectl -n tidb-admin logs tidb-controller-manager-55b887bdc9-lzdwv
Output
... I0113 02:50:13.195779 1 main.go:69] FLAG: --selector="user=dev" ... I0113 02:50:32.409378 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-1/basic1] updated successfully I0113 02:50:32.773635 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-1/basic1] updated successfully I0113 02:51:00.294241 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-1/basic1] updated successfully
View the log of
tidb-controller-manager
of the second TiDB Operator:kubectl -n tidb-admin logs tidb-controller-manager-tidb-operator-v2-5dfcd7f9-vll4c
Output
... I0113 02:50:13.195779 1 main.go:69] FLAG: --selector="user=qa" ... I0113 03:38:43.859387 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-2/basic2] updated successfully I0113 03:38:45.060028 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-2/basic2] updated successfully I0113 03:38:46.261045 1 tidbcluster_control.go:69] TidbCluster: [tidb-cluster-2/basic2] updated successfully
By comparing the logs of the two sets of TiDB Operator, you can confirm that the first TiDB Operator only manages the
tidb-cluster-1/basic1
cluster, and the second TiDB Operator only manages thetidb-cluster-2/basic2
cluster.