TiDB 7.1.1 Release Notes

Release date: July 24, 2023

TiDB version: 7.1.1

Quick access: Quick start | Production deployment

Compatibility changes

• TiDB introduces a new system variable tidb_lock_unchanged_keys to control whether to lock unchanged keys #44714 @ekexium

Behavior changes

• When processing update event, TiCDC splits an event into delete and insert events if the primary key or non-null unique index value is modified in the event. For more information, see documentation.

Improvements

• TiDB

  • Plan Cache supports queries with more than 200 parameters #44823 @qw4990
  • Optimize the performance of reading the dumped chunks from disk #45125 @YangKeao
  • Optimize the logic of constructing index scan range so that it supports converting complex conditions into index scan range #41572 #44389 @xuyifangreeneyes
  • When the retry leader of stale read encounters a lock, TiDB forcibly retries with the leader after resolving the lock, which avoids unnecessary overhead #43659 @you06

• PD

  • PD blocks Swagger API by default when the Swagger server is disabled #6786 @bufferflies

• Tools

  • TiCDC

    • Optimize the encoding format of binary fields when TiCDC replicates data to object storage services #9373 @CharlesCheung96
    • Support the OAUTHBEARER authentication in the scenario of replication to Kafka #8865 @hi-rustin

  • TiDB Lightning

    • Improve the retry logic of TiDB Lightning for the PD ClientTSOStreamClosed error during the checksum phase #45301 @lance6716
    • Verify checksum through SQL after the import to improve stability of verification #41941 @GMHDBJD

  • Dumpling

    • Dumpling avoids executing table queries when the --sql parameter is used, thereby reducing the export overhead #45239 @lance6716

  • TiDB Binlog

    • Optimize the method of retrieving table information to reduce the initialization time and memory usage of Drainer #1137 @lichunzhu

Bug fixes

• TiDB

  • Fix the issue that the GC Resolve Locks step might miss some pessimistic locks #45134 @MyonKeminta
  • Fix the issue that Stats Collector might cause deadlock when a new session is created #44502 @xuyifangreeneyes
  • Fix the potential memory leak issue in memory tracker #44612 @wshwsh12
  • Fix the issue that batch coprocessor retry might generate incorrect Region information that causes query failure #44622 @windtalker
  • Fix the potential data race issue in index scan #45126 @wshwsh12
  • Fix the issue that query results in MPP mode are incorrect when tidb_enable_parallel_apply is enabled #45299 @windtalker
  • Fix the hang-up issue that occurs when queries with indexMerge are killed #45279 @xzhangxian1008
  • Fix the issue that excessive memory consumption of SQL execution details in statistics causes TiDB OOM in extreme cases #44047 @wshwsh12
  • Fix the issue that the FormatSQL() method cannot properly truncate extremely long SQL statements in input #44542 @hawkingrei
  • Fix the issue that DDL operations get stuck during cluster upgrade, which causes upgrade failure #44158 @zimulala
  • Fix the issue that other TiDB nodes do not take over TTL tasks after failures in one TiDB node #45022 @lcwangchao
  • Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the tidb_mem_quota_query limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory #43233 @YangKeao
  • Fix the issue that users can view information in the INFORMATION_SCHEMA.TIFLASH_REPLICA table even without permissions #45320 @Lloyd-Pottiger
  • Fix the issue that ROW_COUNT returned by the ADMIN SHOW DDL JOBS statement is inaccurate #44044 @tangenta
  • Fix the issue that querying a Range COLUMNS partitioned table might get an error #43459 @mjonss
  • Fix the issue that resuming a paused DDL task fails #44217 @dhysum
  • Fix the issue that in-memory pessimistic locks cause FLASHBACK failures and data inconsistency #44292 @JmPotato
  • Fix the issue that deleted tables can still be read from INFORMATION_SCHEMA #43714 @tangenta
  • Fix the issue that the cluster upgrade fails when there are paused DDL operations before the upgrade #44225 @zimulala
  • Fix the duplicate entry error that occurs when restoring a table with AUTO_ID_CACHE=1 using BR #44716 @tiancaiamao
  • Fix the data index inconsistency issue triggered by multiple switches of DDL owner #44619 @tangenta
  • Fix the issue that canceling an ADD INDEX DDL task in the none status might cause memory leak because this task is not removed from the Distributed eXecution Framework (DXF) task queue #44205 @tangenta
  • Fix the issue that the proxy protocol reports the Header read timeout error when processing certain erroneous data #43205 @blacktear23
  • Fix the issue that PD isolation might block the running DDL #44267 @wjhuang2016
  • Fix the issue that the query result of the SELECT CAST(n AS CHAR) statement is incorrect when n in the statement is a negative number #44786 @xhebox
  • Fix the issue of excessive memory usage after creating a large number of empty partitioned tables #44308 @hawkingrei
  • Fix the issue that Join Reorder might cause incorrect outer join results #44314 @AilinKid
  • Fix the issue that queries containing Common Table Expressions (CTEs) might cause insufficient disk space #44477 @guo-shaoge
  • Fix the issue that dropping a database causes slow GC progress #33069 @tiancaiamao
  • Fix the issue that adding an index fails in the ingest mode #44137 @tangenta
  • Fix the issue that the SELECT statement returns an error for a partitioned table if the table partition definition uses the FLOOR() function to round a partitioned column #42323 @jiyfhust
  • Fix the issue that follower read does not handle flashback errors before retrying, which causes query errors #43673 @you06
  • Fix the issue that using memTracker with cursor fetch causes memory leaks #44254 @YangKeao
  • Fix the issue that the SHOW PROCESSLIST statement cannot display the TxnStart of the transaction of the statement with a long subquery time #40851 @crazycs520
  • Fix the issue that the LEADING hint does not support querying block aliases #44645 @qw4990
  • Fix the issue that PREPARE stmt FROM "ANALYZE TABLE xxx" might be killed by tidb_mem_quota_query #44320 @chrysan
  • Fix the panic issue caused by empty processInfo #43829 @zimulala
  • Fix the issue that data and indexes are inconsistent when the ON UPDATE statement does not correctly update the primary key #44565 @zyguan
  • Fix the issue that queries might return incorrect results when tidb_opt_agg_push_down is enabled #44795 @AilinKid
  • Fix the issue that using CTEs and correlated subqueries simultaneously might result in incorrect query results or panic #44649 #38170 #44774 @winoros @guo-shaoge
  • Fix the issue that canceling a DDL task in the rollback state causes errors in related metadata #44143 @wjhuang2016
  • Fix the issue that executing the UPDATE statement causes errors due to checking foreign key constraints #44848 @crazycs520

• PD

  • Fix the issue that Resource Manager repeatedly initializes the default resource group #6787 @glorv
  • Fix the issue that in some cases, the location-labels set in the Placement Rules in SQL does not schedule as expected #6662 @rleungx
  • Fix the issue that redundant replicas cannot be automatically repaired in some corner cases #6573 @nolouch

• TiFlash

  • Fix the issue that in the disaggregated storage and compute architecture mode, the TiFlash compute node fetches inaccurate CPU core information #7436 @guo-shaoge
  • Fix the issue that TiFlash takes too long to restart after using Online Unsafe Recovery #7671 @hongyunyan

• Tools

  • Backup & Restore (BR)

    • Fix the issue that checksum mismatch is falsely reported in some cases #44472 @Leavrth
    • Fix the issue that the frequency of resolve lock is too high when there is no PITR backup task in the TiDB cluster #40759 @joccau

  • TiCDC

    • Fix the issue that a PD exception might cause the replication task to get stuck #8808 #9054 @asddongmen @fubinzh
    • Fix the issue of excessive memory consumption when replicating to an object storage service #8894 @CharlesCheung96
    • Fix the issue that the replication task might get stuck when the redo log is enabled and there is an exception downstream #9172 @CharlesCheung96
    • Fix the issue that TiCDC keeps retrying when there is a downstream failure, which causes the retry time to be too long #9272 @asddongmen
    • Fix the issue of excessive downstream pressure caused by reading downstream metadata too frequently when replicating data to Kafka #8959 @hi-rustin
    • Fix the issue that when the downstream is Kafka, TiCDC queries the downstream metadata too frequently and causes excessive workload in the downstream #8957 #8959 @hi-rustin
    • Fix the OOM issue caused by excessive memory usage of the sorter component in some special scenarios #8974 @hicqu
    • Fix the issue that the UPDATE operation cannot output old values when the Avro or CSV protocol is used #9086 @3AceShowHand
    • Fix the issue that when replicating data to storage services, the JSON file corresponding to downstream DDL statements does not record the default values of table fields #9066 @CharlesCheung96
    • Fix the issue of too many downstream logs caused by frequently setting the downstream bidirectional replication-related variables when replicating data to TiDB or MySQL #9180 @asddongmen
    • Fix the issue that when a replication error occurs due to an oversized Kafka message, the message body is recorded in the log #9031 @darraes
    • Fix the issue that TiCDC gets stuck when PD fails such as network isolation or PD Owner node reboot #8808 #8812 #8877 @asddongmen
    • Fix the issue that Avro protocol incorrectly identifies Enum type values #9259 @3AceShowHand

  • TiDB Data Migration (DM)

    • Fix the issue that DM-master exits abnormally when a unique index contains empty columns in the table to be migrated #9247 @lance6716

  • TiDB Lightning

    • Fix the issue that the failed connection between TiDB Lightning and PD cannot be retried, improving the import success rate #43400 @lichunzhu
    • Fix the issue that TiDB Lightning does not correctly display the error message when writing data to TiKV returns an out of space error #44733 @lance6716
    • Fix the issue that the Region is unavailable error is reported during checksum operation #45462 @D3Hunter
    • Fix the TiDB Lightning panic issue when experimental.allow-expression-index is enabled and the default value is UUID #44497 @lichunzhu
    • Fix the issue that disk quota might be inaccurate due to competing conditions #44867 @D3Hunter
    • Fix the issue that in Logical Import Mode, deleting tables downstream during import might cause TiDB Lightning metadata not to be updated in time #44614 @dsdashun

  • Dumpling

    • Fix the issue that Dumpling exits abnormally when the query result set of --sql is empty #45200 @D3Hunter

  • TiDB Binlog

    • Fix the issue that TiDB cannot correctly query Binlog node status via SHOW PUMP STATUS or SHOW DRAINER STATUS after a complete change of the PD address #42643 @lichunzhu
    • Fix the issue that TiDB cannot write binlogs after a complete change of the PD address #42643 @lance6716
    • Fix the issue that the etcd client does not automatically synchronize the latest node information during initialization #1236 @lichunzhu