TiDB 7.1.1 Release Notes

Release date: July 24, 2023

TiDB version: 7.1.1

Quick access: Quick start | Production deployment

Compatibility changes

• TiDB introduces a new system variable tidb_lock_unchanged_keys to control whether to lock unchanged keys #44714 @ekexium

Behavior changes

• When processing update event, TiCDC splits an event into delete and insert events if the primary key or non-null unique index value is modified in the event. For more information, see documentation.

Improvements

• TiDB

  • Plan Cache supports queries with more than 200 parameters #44823 @qw4990
  • Optimize the performance of reading the dumped chunks from disk #45125 @YangKeao
  • Optimize the logic of constructing index scan range so that it supports converting complex conditions into index scan range #41572 #44389 @xuyifangreeneyes
  • When the retry leader of stale read encounters a lock, TiDB forcibly retries with the leader after resolving the lock, which avoids unnecessary overhead #43659 @you06

• PD

  • PD blocks Swagger API by default when the Swagger server is disabled #6786 @bufferflies

• Tools

  • TiCDC

    • Optimize the encoding format of binary fields when TiCDC replicates data to object storage services #9373 @CharlesCheung96
    • Support the OAUTHBEARER authentication in the scenario of replication to Kafka #8865 @hi-rustin

  • TiDB Lightning

    • Improve the retry logic of TiDB Lightning for the PD ClientTSOStreamClosed error during the checksum phase #45301 @lance6716
    • Verify checksum through SQL after the import to improve stability of verification #41941 @GMHDBJD

  • Dumpling

    • Dumpling avoids executing table queries when the --sql parameter is used, thereby reducing the export overhead #45239 @lance6716

  • TiDB Binlog

    • Optimize the method of retrieving table information to reduce the initialization time and memory usage of Drainer #1137 @lichunzhu

Bug fixes

• TiDB

  • Fix the issue that the GC Resolve Locks step might miss some pessimistic locks #45134 @MyonKeminta
  • Fix the issue that Stats Collector might cause deadlock when a new session is created #44502 @xuyifangreeneyes
  • Fix the potential memory leak issue in memory tracker #44612 @wshwsh12
  • Fix the issue that batch coprocessor retry might generate incorrect Region information that causes query failure #44622 @windtalker
  • Fix the potential data race issue in index scan #45126 @wshwsh12
  • Fix the issue that query results in MPP mode are incorrect when tidb_enable_parallel_apply is enabled #45299 @windtalker
  • Fix the hang-up issue that occurs when queries with indexMerge are killed #45279 @xzhangxian1008
  • Fix the issue that excessive memory consumption of SQL execution details in statistics causes TiDB OOM in extreme cases #44047 @wshwsh12
  • Fix the issue that the FormatSQL() method cannot properly truncate extremely long SQL statements in input #44542 @hawkingrei
  • Fix the issue that DDL operations get stuck during cluster upgrade, which causes upgrade failure #44158 @zimulala
  • Fix the issue that other TiDB nodes do not take over TTL tasks after failures in one TiDB node #45022 @lcwangchao
  • Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the tidb_mem_quota_query limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory #43233 @YangKeao
  • Fix the issue that users can view information in the INFORMATION_SCHEMA.TIFLASH_REPLICA table even without permissions #45320 @Lloyd-Pottiger
  • Fix the issue that ROW_COUNT returned by the ADMIN SHOW DDL JOBS statement is inaccurate #44044 @tangenta
  • Fix the issue that querying a Range COLUMNS partitioned table might get an error #43459 @mjonss
  • Fix the issue that resuming a paused DDL task fails #44217 @dhysum
  • Fix the issue that in-memory pessimistic locks cause FLASHBACK failures and data inconsistency #44292 @JmPotato
  • Fix the issue that deleted tables can still be read from INFORMATION_SCHEMA #43714 @tangenta
  • Fix the issue that the cluster upgrade fails when there are paused DDL operations before the upgrade #44225 @zimulala
  • Fix the duplicate entry error that occurs when restoring a table with AUTO_ID_CACHE=1 using BR #44716 @tiancaiamao
  • Fix the data index inconsistency issue triggered by multiple switches of DDL owner #44619 @tangenta
  • Fix the issue that canceling an ADD INDEX DDL task in the none status might cause memory leak because this task is not removed from the backend task queue #44205 @tangenta
  • Fix the issue that the proxy protocol reports the Header read timeout error when processing certain erroneous data #43205 @blacktear23
  • Fix the issue that PD isolation might block the running DDL #44267 @wjhuang2016
  • Fix the issue that the query result of the SELECT CAST(n AS CHAR) statement is incorrect when n in the statement is a negative number #44786 @xhebox
  • Fix the issue of excessive memory usage after creating a large number of empty partitioned tables #44308 @hawkingrei
  • Fix the issue that Join Reorder might cause incorrect outer join results #44314 @AilinKid
  • Fix the issue that queries containing Common Table Expressions (CTEs) might cause insufficient disk space #44477 @guo-shaoge
  • Fix the issue that dropping a database causes slow GC progress #33069 @tiancaiamao
  • Fix the issue that adding an index fails in the ingest mode #44137 @tangenta
  • Fix the issue that the SELECT statement returns an error for a partitioned table if the table partition definition uses the FLOOR() function to round a partitioned column #42323 @jiyfhust
  • Fix the issue that follower read does not handle flashback errors before retrying, which causes query errors #43673 @you06
  • Fix the issue that using memTracker with cursor fetch causes memory leaks #44254 @YangKeao
  • Fix the issue that the SHOW PROCESSLIST statement cannot display the TxnStart of the transaction of the statement with a long subquery time #40851 @crazycs520
  • Fix the issue that the LEADING hint does not support querying block aliases #44645 @qw4990
  • Fix the issue that PREPARE stmt FROM "ANALYZE TABLE xxx" might be killed by tidb_mem_quota_query #44320 @chrysan
  • Fix the panic issue caused by empty processInfo #43829 @zimulala
  • Fix the issue that data and indexes are inconsistent when the ON UPDATE statement does not correctly update the primary key #44565 @zyguan
  • Fix the issue that queries might return incorrect results when tidb_opt_agg_push_down is enabled #44795 @AilinKid
  • Fix the issue that using CTEs and correlated subqueries simultaneously might result in incorrect query results or panic #44649 #38170 #44774 @winoros @guo-shaoge
  • Fix the issue that canceling a DDL task in the rollback state causes errors in related metadata #44143 @wjhuang2016
  • Fix the issue that executing the UPDATE statement causes errors due to checking foreign key constraints #44848 @crazycs520

• PD

  • Fix the issue that Resource Manager repeatedly initializes the default resource group #6787 @glorv
  • Fix the issue that in some cases, the location-labels set in the Placement Rules in SQL does not schedule as expected #6662 @rleungx
  • Fix the issue that redundant replicas cannot be automatically repaired in some corner cases #6573 @nolouch

• TiFlash

  • Fix the issue that in the disaggregated storage and compute architecture mode, the TiFlash compute node fetches inaccurate CPU core information #7436 @guo-shaoge
  • Fix the issue that TiFlash takes too long to restart after using Online Unsafe Recovery #7671 @hongyunyan

• Tools

  • Backup & Restore (BR)

    • Fix the issue that checksum mismatch is falsely reported in some cases #44472 @Leavrth
    • Fix the issue that the frequency of resolve lock is too high when there is no PITR backup task in the TiDB cluster #40759 @joccau

  • TiCDC

    • Fix the issue that a PD exception might cause the replication task to get stuck #8808 #9054 @asddongmen @fubinzh
    • Fix the issue of excessive memory consumption when replicating to an object storage service #8894 @CharlesCheung96
    • Fix the issue that the replication task might get stuck when the redo log is enabled and there is an exception downstream #9172 @CharlesCheung96
    • Fix the issue that TiCDC keeps retrying when there is a downstream failure, which causes the retry time to be too long #9272 @asddongmen
    • Fix the issue of excessive downstream pressure caused by reading downstream metadata too frequently when replicating data to Kafka #8959 @hi-rustin
    • Fix the issue that when the downstream is Kafka, TiCDC queries the downstream metadata too frequently and causes excessive workload in the downstream #8957 #8959 @hi-rustin
    • Fix the OOM issue caused by excessive memory usage of the sorter component in some special scenarios #8974 @hicqu
    • Fix the issue that the UPDATE operation cannot output old values when the Avro or CSV protocol is used #9086 @3AceShowHand
    • Fix the issue that when replicating data to storage services, the JSON file corresponding to downstream DDL statements does not record the default values of table fields #9066 @CharlesCheung96
    • Fix the issue of too many downstream logs caused by frequently setting the downstream bidirectional replication-related variables when replicating data to TiDB or MySQL #9180 @asddongmen
    • Fix the issue that when a replication error occurs due to an oversized Kafka message, the message body is recorded in the log #9031 @darraes
    • Fix the issue that TiCDC gets stuck when PD fails such as network isolation or PD Owner node reboot #8808 #8812 #8877 @asddongmen
    • Fix the issue that Avro protocol incorrectly identifies Enum type values #9259 @3AceShowHand

  • TiDB Data Migration (DM)

    • Fix the issue that DM-master exits abnormally when a unique index contains empty columns in the table to be migrated #9247 @lance6716

  • TiDB Lightning

    • Fix the issue that the failed connection between TiDB Lightning and PD cannot be retried, improving the import success rate #43400 @lichunzhu
    • Fix the issue that TiDB Lightning does not correctly display the error message when writing data to TiKV returns an out of space error #44733 @lance6716
    • Fix the issue that the Region is unavailable error is reported during checksum operation #45462 @D3Hunter
    • Fix the TiDB Lightning panic issue when experimental.allow-expression-index is enabled and the default value is UUID #44497 @lichunzhu
    • Fix the issue that disk quota might be inaccurate due to competing conditions #44867 @D3Hunter
    • Fix the issue that in Logical Import Mode, deleting tables downstream during import might cause TiDB Lightning metadata not to be updated in time #44614 @dsdashun

  • Dumpling

    • Fix the issue that Dumpling exits abnormally when the query result set of --sql is empty #45200 @D3Hunter

  • TiDB Binlog

    • Fix the issue that TiDB cannot correctly query Binlog node status via SHOW PUMP STATUS or SHOW DRAINER STATUS after a complete change of the PD address #42643 @lichunzhu
    • Fix the issue that TiDB cannot write binlogs after a complete change of the PD address #42643 @lance6716
    • Fix the issue that the etcd client does not automatically synchronize the latest node information during initialization #1236 @lichunzhu