- Introduction
- Get Started
- Deploy
- In Self-Managed Kubernetes
- In Public Cloud Kubernetes
- Deploy TiDB on ARM64 Machines
- Deploy TiFlash to Explore TiDB HTAP
- Deploy TiDB Across Multiple Kubernetes Clusters
- Deploy a Heterogeneous TiDB Cluster
- Deploy TiCDC
- Deploy TiDB Binlog
- Monitor and Alert
- Migrate
- Import Data
- Migrate from MySQL
- Migrate TiDB to Kubernetes
- Manage
- Secure
- Scale
- Upgrade
- Upgrade a TiDB Cluster
- Upgrade TiDB Operator
- Backup and Restore
- Overview
- Backup and Restore Custom Resources
- Grant Permissions to Remote Storage
- Amazon S3 Compatible Storage
- Google Cloud Storage
- Azure Blob Storage
- Persistent Volumes
- Maintain
- Restart a TiDB Cluster
- Destroy a TiDB Cluster
- View TiDB Logs
- Modify TiDB Cluster Configuration
- Configure Automatic Failover
- Pause Sync of TiDB Cluster
- Maintain Different TiDB Clusters Separately Using Multiple TiDB Operator
- Maintain Kubernetes Nodes
- Migrate from Helm 2 to Helm 3
- Replace Nodes for a TiDB Cluster
- Disaster Recovery
- Troubleshoot
- FAQs
- Reference
- Release Notes
- v1.3
- v1.2
- v1.1
- v1.0
- v0
Restore Data from S3-Compatible Storage Using BR
This document describes how to restore the TiDB cluster data backed up using TiDB Operator in Kubernetes.
The restore method described in this document is implemented based on CustomResourceDefinition (CRD) in TiDB Operator. For the underlying implementation, BR is used to restore the data. BR stands for Backup & Restore, which is a command-line tool for distributed backup and recovery of the TiDB cluster data.
Usage Scenarios
After backing up TiDB cluster data to Amazon S3 using BR, if you need to recover the backup SST (key-value pairs) files from Amazon S3 to a TiDB cluster, you can follow steps in this document to restore the data using BR.
- BR is only applicable to TiDB v3.1 or later releases.
- Data restored by BR cannot be replicated to a downstream cluster, because BR directly imports SST files to TiDB and the downstream cluster currently cannot access the upstream SST files.
This document provides an example about how to restore the backup data from the spec.s3.prefix
folder of the spec.s3.bucket
bucket on Amazon S3 to the demo2
TiDB cluster in the test2
namespace. The following are the detailed steps.
Step 1: Prepare the restore environment
Before restoring backup data on a S3-compatible storage to TiDB using BR, take the following steps to prepare the restore environment:
Download backup-rbac.yaml, and execute the following command to create the role-based access control (RBAC) resources in the
test2
namespace:kubectl apply -f backup-rbac.yaml -n test2
Grant permissions to the remote storage.
If the data to be restored is in Amazon S3, you can grant permissions in three methods. For more information, see AWS account permissions.
If the data to be restored is in other S3-compatible storage (such as Ceph and MinIO), you can grant permissions by using AccessKey and SecretKey.
For a TiDB version earlier than v4.0.8, you also need to complete the following preparation steps. For TiDB v4.0.8 or a later version, skip these preparation steps.
Make sure that you have the
SELECT
andUPDATE
privileges on themysql.tidb
table of the target database so that theRestore
CR can adjust the GC time before and after the restore.Create the
restore-demo2-tidb-secret
secret to store the account and password to access the TiDB cluster:kubectl create secret generic restore-demo2-tidb-secret --from-literal=password=${password} --namespace=test2
Step 2: Restore the backup data to a TiDB cluster
Depending on which method you choose to grant permissions to the remote storage when preparing the restore environment, you can restore the data by doing one of the following:
Method 1: If you grant permissions by importing AccessKey and SecretKey, create the
Restore
CR to restore cluster data as described below:kubectl apply -f resotre-aws-s3.yaml
The content of
restore-aws-s3.yaml
is as follows:--- apiVersion: pingcap.com/v1alpha1 kind: Restore metadata: name: demo2-restore-s3 namespace: test2 spec: br: cluster: demo2 clusterNamespace: test2 # logLevel: info # statusAddr: ${status_addr} # concurrency: 4 # rateLimit: 0 # timeAgo: ${time} # checksum: true # sendCredToTikv: true # # Only needed for TiDB Operator < v1.1.10 or TiDB < v4.0.8 # to: # host: ${tidb_host} # port: ${tidb_port} # user: ${tidb_user} # secretName: restore-demo2-tidb-secret s3: provider: aws secretName: s3-secret region: us-west-1 bucket: my-bucket prefix: my-folder
Method 2: If you grant permissions by associating IAM with Pod, create the
Restore
CR to restore cluster data as described below:kubectl apply -f restore-aws-s3.yaml
The content of
restore-aws-s3.yaml
is as follows:--- apiVersion: pingcap.com/v1alpha1 kind: Restore metadata: name: demo2-restore-s3 namespace: test2 annotations: iam.amazonaws.com/role: arn:aws:iam::123456789012:role/user spec: br: cluster: demo2 sendCredToTikv: false clusterNamespace: test2 # logLevel: info # statusAddr: ${status_addr} # concurrency: 4 # rateLimit: 0 # timeAgo: ${time} # checksum: true # Only needed for TiDB Operator < v1.1.10 or TiDB < v4.0.8 to: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: restore-demo2-tidb-secret s3: provider: aws region: us-west-1 bucket: my-bucket prefix: my-folder
Method 3: If you grant permissions by associating IAM with ServiceAccount, create the
Restore
CR to restore cluster data as described below:kubectl apply -f restore-aws-s3.yaml
The content of
restore-aws-s3.yaml
is as follows:--- apiVersion: pingcap.com/v1alpha1 kind: Restore metadata: name: demo2-restore-s3 namespace: test2 spec: serviceAccount: tidb-backup-manager br: cluster: demo2 sendCredToTikv: false clusterNamespace: test2 # logLevel: info # statusAddr: ${status_addr} # concurrency: 4 # rateLimit: 0 # timeAgo: ${time} # checksum: true # Only needed for TiDB Operator < v1.1.10 or TiDB < v4.0.8 to: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: restore-demo2-tidb-secret s3: provider: aws region: us-west-1 bucket: my-bucket prefix: my-folder
When configuring restore-aws-s3.yaml
, note the following:
- For more information about S3-compatible storage configuration, refer to S3 storage fields.
- Some parameters in
.spec.br
are optional, such aslogLevel
,statusAddr
,concurrency
,rateLimit
,checksum
,timeAgo
, andsendCredToTikv
. For more information about BR configuration, refer to BR fields. - For v4.0.8 or a later version, BR can automatically adjust
tikv_gc_life_time
. You do not need to configurespec.to
fields in theRestore
CR. - For more information about the
Restore
CR fields, refer to Restore CR fields.
After creating the Restore
CR, execute the following command to check the restore status:
kubectl get rt -n test2 -o wide
Troubleshooting
If you encounter any problem during the restore process, refer to Common Deployment Failures.