How to Run TPC-C Test on TiDB

This document describes how to test TiDB using TPC-C.

TPC-C is an online transaction processing (OLTP) benchmark. It tests the OLTP system by using a commodity sales model that involves the following five transactions of different types:

NewOrder
Payment
OrderStatus
Delivery
StockLevel

Prepare

Before testing, TPC-C Benchmark specifies the initial state of the database, which is the rule for data generation in the database. The ITEM table contains a fixed number of 100,000 items, while the number of warehouses can be adjusted. If there are W records in the WAREHOUSE table, then:

The STOCK table has W * 100,000 records (Each warehouse corresponds to the stock data of 100,000 items)
The DISTRICT table has W * 10 records (Each warehouse provides services to 10 districts)
The CUSTOMER table has W * 10 * 3,000 records (Each district has 3,000 customers)
The HISTORY table has W * 10 * 3,000 records (Each customer has one transaction history)
The ORDER table has W * 10 * 3,000 records (Each district has 3,000 orders and the last 900 orders generated are added to the NEW-ORDER table. Each order randomly generates 5 ~ 15 ORDER-LINE records.

In this document, the testing uses 1,000 warehouses as an example to test TiDB.

TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute.

The test in this document is implemented based on go-tpc. You can download the test program using TiUP commands.

tiup install bench

For detailed usage of the TiUP Bench component, see TiUP Bench.

Load data

Loading data is usually the most time-consuming and problematic stage of the entire TPC-C test. This section provides the following command to load data.

Execute the following TiUP command in Shell:

tiup bench tpcc -H 172.16.5.140 -P 4000 -D tpcc --warehouses 1000 prepare

Based on different machine configurations, this loading process might take a few hours. If the cluster size is small, you can use a smaller WAREHOUSE value for the test.

After the data is loaded, you can execute the tiup bench tpcc -H 172.16.5.140 -P 4000 -D tpcc --warehouses 4 check command to validate the data correctness.

Run the test

Execute the following command to run the test:

tiup bench tpcc -H 172.16.5.140 -P 4000 -D tpcc --warehouses 1000 run

During the test, test results are continuously printed on the console:

[Current] NEW_ORDER - Takes(s): 4.6, Count: 5, TPM: 65.5, Sum(ms): 4604, Avg(ms): 920, 90th(ms): 1500, 99th(ms): 1500, 99.9th(ms): 1500
[Current] ORDER_STATUS - Takes(s): 1.4, Count: 1, TPM: 42.2, Sum(ms): 256, Avg(ms): 256, 90th(ms): 256, 99th(ms): 256, 99.9th(ms): 256
[Current] PAYMENT - Takes(s): 6.9, Count: 5, TPM: 43.7, Sum(ms): 2208, Avg(ms): 441, 90th(ms): 512, 99th(ms): 512, 99.9th(ms): 512
[Current] STOCK_LEVEL - Takes(s): 4.4, Count: 1, TPM: 13.8, Sum(ms): 224, Avg(ms): 224, 90th(ms): 256, 99th(ms): 256, 99.9th(ms): 256
...

After the test is finished, the test summary results are printed:

[Summary] DELIVERY - Takes(s): 455.2, Count: 32, TPM: 4.2, Sum(ms): 44376, Avg(ms): 1386, 90th(ms): 2000, 99th(ms): 4000, 99.9th(ms): 4000
[Summary] DELIVERY_ERR - Takes(s): 455.2, Count: 1, TPM: 0.1, Sum(ms): 953, Avg(ms): 953, 90th(ms): 1000, 99th(ms): 1000, 99.9th(ms): 1000
[Summary] NEW_ORDER - Takes(s): 487.8, Count: 314, TPM: 38.6, Sum(ms): 282377, Avg(ms): 899, 90th(ms): 1500, 99th(ms): 1500, 99.9th(ms): 1500
[Summary] ORDER_STATUS - Takes(s): 484.6, Count: 29, TPM: 3.6, Sum(ms): 8423, Avg(ms): 290, 90th(ms): 512, 99th(ms): 1500, 99.9th(ms): 1500
[Summary] PAYMENT - Takes(s): 490.1, Count: 321, TPM: 39.3, Sum(ms): 144708, Avg(ms): 450, 90th(ms): 512, 99th(ms): 1000, 99.9th(ms): 1500
[Summary] STOCK_LEVEL - Takes(s): 487.6, Count: 41, TPM: 5.0, Sum(ms): 9318, Avg(ms): 227, 90th(ms): 512, 99th(ms): 1000, 99.9th(ms): 1000

After the test is finished, you can execute the tiup bench tpcc -H 172.16.5.140 -P 4000 -D tpcc --warehouses 4 check command to validate the data correctness.

Clean up test data

Execute the following command to clean up the test data:

tiup bench tpcc -H 172.16.5.140 -P 4000 -D tpcc --warehouses 4 cleanup