- Introduction
- Concepts
- Architecture
- Key Features
- Horizontal Scalability
- MySQL Compatible Syntax
- Replicate from and to MySQL
- Distributed Transactions with Strong Consistency
- Cloud Native Architecture
- Minimize ETL with HTAP
- Fault Tolerance & Recovery with Raft
- Automatic Rebalancing
- Deployment and Orchestration with Ansible, Kubernetes, Docker
- JSON Support
- Spark Integration
- Read Historical Data Without Restoring from Backup
- Fast Import and Restore of Data
- Hybrid of Column and Row Storage
- SQL Plan Management
- Open Source
- Online Schema Changes
- How-to
- Get Started
- Deploy
- Hardware Recommendations
- From Binary Tarball
- Orchestrated Deployment
- Geographic Redundancy
- Data Migration with Ansible
- Configure
- Secure
- Transport Layer Security (TLS)
- Generate Self-signed Certificates
- Monitor
- Migrate
- Maintain
- Common Ansible Operations
- Backup and Restore
- Use BR (recommended)
- Identify Abnormal Queries
- Scale
- Upgrade
- Troubleshoot
- Reference
- SQL
- MySQL Compatibility
- SQL Language Structure
- Attributes
- Data Types
- Functions and Operators
- Function and Operator Reference
- Type Conversion in Expression Evaluation
- Operators
- Control Flow Functions
- String Functions
- Numeric Functions and Operators
- Date and Time Functions
- Bit Functions and Operators
- Cast Functions and Operators
- Encryption and Compression Functions
- Information Functions
- JSON Functions
- Aggregate (GROUP BY) Functions
- Window Functions
- Miscellaneous Functions
- Precision Math
- List of Expressions for Pushdown
- SQL Statements
ADD COLUMN
ADD INDEX
ADMIN
ADMIN CANCEL DDL
ADMIN CHECKSUM TABLE
ADMIN CHECK [TABLE|INDEX]
ADMIN SHOW DDL [JOBS|QUERIES]
ALTER DATABASE
ALTER INSTANCE
ALTER TABLE
ALTER USER
ANALYZE TABLE
BEGIN
CHANGE COLUMN
COMMIT
CREATE DATABASE
CREATE INDEX
CREATE ROLE
CREATE TABLE LIKE
CREATE TABLE
CREATE USER
CREATE VIEW
DEALLOCATE
DELETE
DESC
DESCRIBE
DO
DROP COLUMN
DROP DATABASE
DROP INDEX
DROP ROLE
DROP TABLE
DROP USER
DROP VIEW
EXECUTE
EXPLAIN ANALYZE
EXPLAIN
FLUSH PRIVILEGES
FLUSH STATUS
FLUSH TABLES
GRANT <privileges>
GRANT <role>
INSERT
KILL [TIDB]
LOAD DATA
LOAD STATS
MODIFY COLUMN
PREPARE
RECOVER TABLE
RENAME INDEX
RENAME TABLE
REPLACE
REVOKE <privileges>
REVOKE <role>
ROLLBACK
SELECT
SET DEFAULT ROLE
SET [NAMES|CHARACTER SET]
SET PASSWORD
SET ROLE
SET TRANSACTION
SET [GLOBAL|SESSION] <variable>
SHOW ANALYZE STATUS
SHOW CHARACTER SET
SHOW COLLATION
SHOW [FULL] COLUMNS FROM
SHOW CREATE TABLE
SHOW CREATE USER
SHOW DATABASES
SHOW ENGINES
SHOW ERRORS
SHOW [FULL] FIELDS FROM
SHOW GRANTS
SHOW INDEXES [FROM|IN]
SHOW INDEX [FROM|IN]
SHOW KEYS [FROM|IN]
SHOW PRIVILEGES
SHOW [FULL] PROCESSSLIST
SHOW SCHEMAS
SHOW STATUS
SHOW [FULL] TABLES
SHOW TABLE REGIONS
SHOW TABLE STATUS
SHOW [GLOBAL|SESSION] VARIABLES
SHOW WARNINGS
SPLIT REGION
START TRANSACTION
TRACE
TRUNCATE
UPDATE
USE
- Constraints
- Generated Columns
- Partitioning
- Character Set
- SQL Mode
- Views
- Configuration
- Security
- Transactions
- System Databases
- Errors Codes
- Supported Client Drivers
- Garbage Collection (GC)
- Performance
- Overview
- Understanding the Query Execution Plan
- The Blocklist of Optimization Rules and Expression Pushdown
- Introduction to Statistics
- TopN and Limit Push Down
- Optimizer Hints
- Follower Read
- Check the TiDB Cluster Status Using SQL Statements
- Execution Plan Binding
- Statement Summary Table
- Tune TiKV
- Operating System Tuning
- Column Pruning
- Key Monitoring Metrics
- Alert Rules
- Best Practices
- TiSpark
- TiKV
- TiFlash
- TiDB Binlog
- Tools
- Overview
- Use Cases
- Download
- TiDB Operator
- Table Filter
- Backup & Restore (BR)
- Mydumper
- Syncer
- Loader
- Data Migration
- TiDB Lightning
- sync-diff-inspector
- PD Control
- PD Recover
- TiKV Control
- TiDB Control
- TiDB in Kubernetes
- FAQs
- Support
- Contribute
- Adopters
- Releases
- All Releases
- v3.1
- v3.0
- v2.1
- v2.0
- v1.0
- Glossary
You are viewing the documentation of an older version of the TiDB database (TiDB v3.1).
Deploy a TiFlash Cluster
If you want to get a first-hand experience on how to use the TiFlash RC version, contact PingCAP for more information and assistance.
This document introduces the environment requirements for deploying a TiFlash cluster and the deployment methods in different scenarios.
Recommended hardware configuration
This section provides hardware configuration recommendations based on different TiFlash deployment methods.
TiFlash standalone deployment
- Minimum configuration: 32 VCore, 64 GB RAM, 1 SSD + n HDD
- Recommended configuration: 48 VCore, 128 GB RAM, 1 NVMe SSD + n SSD
There is no limit to the number of deployment machines (one at least). A single machine can use multiple disks, but deploying multiple instances on a single machine is not recommended.
It is recommended to use an SSD disk to buffer the real-time data being replicated and written to TiFlash. The performance of this disk need to be not lower than the hard disk used by TiKV. It is recommended that you use a better performance NVMe SSD and the SSD's capacity is not less than 10% of the total capacity. Otherwise, it might become the bottleneck of the amount of data that this node can handle.
For other hard disks, you can use multiple HDDs or regular SSDs. A better hard disk will surely bring better performance.
TiFlash supports multi-disk deployment, so there is no need to use RAID.
TiFlash and TiKV are deployed on the same node
See Hardware recommendations for TiKV server, and increase the memory capacity and the number of and CPU cores as needed.
It is not recommended to deploy TiFlash and TiKV on the same disk to prevent mutual interference.
Hard disk selection criteria are the same as TiFlash standalone deployment. The total capacity of the hard disk is roughly: the to-be-replicated data capacity of the entire TiKV cluster / the number of TiKV replicas * the number of TiFlash replicas
.
For example, if the overall planned capacity of TiKV is 1TB, each Region in TiKV has 3 replicas, and each Region in TiFlash has 2 replicas, then the recommended capacity of TiFlash will be 1024GB / 3 * 2
. You can choose to replicate part of the tables instead of all.
TiDB version requirements
Currently, the testing of TiFlash is based on the related components of TiDB 3.1 (including TiDB, PD, TiKV, and TiFlash). For the download method of TiDB 3.1, refer to the following installation and deployment steps.
Install and deploy TiFlash
This section describes how to install and deploy TiFlash in the following scenarios:
- Before starting the TiFlash process, you must ensure that PD's Placement Rules feature is enabled (For how to enable it, see the second step in the Add TiFlash component to an existing TiDB cluster section).
- When TiFlash is running, you must ensure that PD's Placement Rules feature remains enabled.
Fresh TiFlash deployment
For fresh TiFlash deployment, it is recommended to deploy TiFlash by downloading an offline installation package. The steps are as follows:
Download TiDB Ansible with the specified tag corresponding to the TiDB 3.1 version:
git clone -b $tag https://github.com/pingcap/tidb-ansible.git
Download the binary files of TiDB, TiKV, PD, and other components:
ansible-playbook local_prepare.yml
Edit the
inventory.ini
configuration file. In addition to configuring for TiDB cluster deployment, you also need to specify the IPs of your TiFlash servers under the[tiflash_servers]
section (currently only IPs are supported; domain names are not supported).If you want to customize the deployment directory, configure the
data_dir
parameter. If you want multi-disk deployment, separate the deployment directories with commas (note that the parent directory of eachdata_dir
directory needs to give thetidb
user write permissions). For example:[tiflash_servers] 192.168.1.1 data_dir=/data1/tiflash/data,/data2/tiflash/data
Complete the remaining steps of the TiDB Ansible deployment process.
To verify that TiFlash has been successfully deployed:
- Execute the
pd-ctl store http://your-pd-address
command in pd-ctl (resources/bin
in the tidb-ansible directory includes the pd-ctl binary file). - Observe that the status of the deployed TiFlash instance is "Up".
- Execute the
Add TiFlash component to an existing TiDB cluster
First, confirm that your current TiDB version supports TiFlash, otherwise you need to upgrade your TiDB cluster to 3.1 rc or higher according to TiDB Upgrade Guide.
Execute the
config set enable-placement-rules true
command in pd-ctl (resources/bin
in the tidb-ansible directory includes the pd-ctl binary file) to enable PD's Placement Rules feature.Edit the
inventory.ini
configuration file. You need to specify the IPs of your TiFlash servers under the[tiflash_servers]
section (currently only IPs are supported; domain names are not supported).If you want to customize the deployment directory, configure the
data_dir
parameter. If you want multi-disk deployment separate the deployment directories with commas (note that the parent directory of eachdata_dir
directory needs to give thetidb
user write permissions). For example:[tiflash_servers] 192.168.1.1 data_dir=/data1/tiflash/data,/data2/tiflash/data
NoteEven if TiFlash and TiKV are deployed on the same machine, TiFlash uses a different default port from TiKV. TiFlash's default port is 9000. If you want to modify the port, add a new line
tcp_port=xxx
to theinventory.ini
configuration file.Execute the following ansible-playbook commands to deploy TiFlash:
ansible-playbook local_prepare.yml && ansible-playbook -t tiflash deploy.yml && ansible-playbook -t tiflash start.yml && ansible-playbook rolling_update_monitor.yml
To verify that TiFlash has been successfully deployed:
- Execute the
pd-ctl store http://your-pd-address
command in pd-ctl (resources/bin
in the tidb-ansible directory includes the pd-ctl binary file). - Observe that the status of the deployed TiFlash instance is "Up".
- Execute the