TiDB 7.5.4 Release Notes

Release date: October 15, 2024

TiDB version: 7.5.4

Quick access: Quick start | Production deployment

Compatibility changes

• Set a default limit of 2048 for DDL historical tasks retrieved through the TiDB HTTP API to prevent OOM issues caused by excessive historical tasks #55711 @joccau

Improvements

• TiDB

  • Support applying the tidb_redact_log setting to the output of EXPLAIN statements and further optimize the logic in processing logs #54565 @hawkingrei
  • Optimize the query speed of TiDB slow queries #54630 @yibin87

• TiKV

  • Optimize the trigger mechanism of RocksDB compaction to accelerate disk space reclamation when handling a large number of DELETE versions #17269 @AndreMouche
  • Reduce the memory usage of the peer message channel #16229 @Connor1996
  • Optimize the jittery access delay when restarting TiKV due to waiting for the log to be applied, improving the stability of TiKV #15874 @LykxSassinator
  • Optimize TiKV's DiskFull detection to make it compatible with RaftEngine's spill-dir configuration, ensuring that this feature works consistently #17356 @LykxSassinator

• TiFlash

  • Optimize the execution efficiency of LENGTH() and ASCII() functions #9344 @xzhangxian1008
  • Mitigate the issue that TiFlash might panic due to updating certificates after TLS is enabled #8535 @windtalker
  • Improve the cancel mechanism of the JOIN operator, so that the JOIN operator can respond to cancel requests in a timely manner #9430 @windtalker

• Tools

  • Backup & Restore (BR)

    • Support checking whether the disk space in TiKV is sufficient before TiKV downloads each SST file. If the space is insufficient, BR terminates the restore and returns an error #17224 @RidRisR

  • TiCDC

    • When the downstream is TiDB with the SUPER permission granted, TiCDC supports querying the execution status of ADD INDEX DDL from the downstream database to avoid data replication failure due to timeout in retrying executing the DDL statement in some cases #10682 @CharlesCheung96

Bug fixes

• TiDB

  • Fix the issue that FLASHBACK DATABASE fails when many tables exist in the database #54415 @lance6716
  • Fix the issue that RANGE partitioned tables that are not strictly self-incrementing can be created #54829 @Defined2014
  • Fix the issue that a query statement that contains UNION might return incorrect results #52985 @XuHuaiyu
  • Fix the issue that INDEX_HASH_JOIN cannot exit properly when SQL is abnormally interrupted #54688 @wshwsh12
  • Reset the parameters in the Open method of PipelinedWindow to fix the unexpected error that occurs when the PipelinedWindow is used as a child node of Apply due to the reuse of previous parameter values caused by repeated opening and closing operations #53600 @XuHuaiyu
  • Fix the issue that the query latency of stale reads increases, caused by information schema cache misses #53428 @crazycs520
  • Fix the issue that disk files might not be deleted after the Sort operator spills and a query error occurs #55061 @wshwsh12
  • Fix the issue that the query might return incorrect results instead of an error after being killed #50089 @D3Hunter
  • Fix the issue that table replication fails when the index length of the table replicated from DM exceeds the maximum length specified by max-index-length #55138 @lance6716
  • Fix the issue that the SUB_PART value in the INFORMATION_SCHEMA.STATISTICS table is NULL #55812 @Defined2014
  • Fix the issue that an error occurs when a DML statement contains nested generated columns #53967 @wjhuang2016
  • Fix the issue that the tot_col_size column in the mysql.stats_histograms table might be a negative number #55126 @qw4990
  • Fix the data race issue in IndexNestedLoopHashJoin #49692 @solotzg
  • Fix the issue that the query might get stuck when terminated because the memory usage exceeds the limit set by tidb_mem_quota_query #55042 @yibin87
  • Fix the issue that columnEvaluator cannot identify the column references in the input chunk, which leads to runtime error: index out of range when executing SQL statements #53713 @AilinKid
  • Fix the issue that the performance of the SELECT ... WHERE ... ORDER BY ... statement execution is poor in some cases #54969 @tiancaiamao
  • Fix the issue that empty groupOffset in StreamAggExec might cause TiDB to panic #53867 @xzhangxian1008
  • Fix the issue that TiDB queries cannot be canceled during cop task construction #55957 @yibin87
  • Fix the issue that an out of range error might occur when a small display width is specified for a column of the integer type #55837 @windtalker
  • Fix the issue that duplicate entry might occur when adding unique indexes #56161 @tangenta
  • Fix the issue that TiDB panics when importing temporary tables using the IMPORT INTO statement #55970 @D3Hunter
  • Fix the issue of data index inconsistency caused by retries during index addition #55808 @lance6716

• TiKV

  • Fix the issue that TiKV might panic when a stale replica processes Raft snapshots, triggered by a slow split operation and immediate removal of the new replica #17469 @hbisheng
  • Fix the flow control issue that might occur after deleting large tables or partitions #17304 @Connor1996
  • Fix the issue that bloom filters are incompatible between earlier versions (earlier than v7.1) and later versions #17272 @v01dstar
  • Fix the issue that prevents master key rotation when the master key is stored in a Key Management Service (KMS) #17410 @hhwyt
  • Fix the issue that the Storage async write duration monitoring metric on the TiKV panel in Grafana is inaccurate #17579 @overvenus
  • Fix the issue that when a large number of transactions are queuing for lock release on the same key and the key is frequently updated, excessive pressure on deadlock detection might cause TiKV OOM issues #17394 @MyonKeminta

• PD

  • Fix the issue that PD's Region API cannot be requested when a large number of Regions exist #55872 @rleungx
  • Fix the issue that when using a wrong parameter in evict-leader-scheduler, PD does not report errors correctly and some schedulers are unavailable #8619 @rleungx
  • Fix the issue that data race might occur in the scheduling server when a PD leader is switched in the microservice mode #8538 @lhy1024
  • Fix the issue that slots are not fully deleted in a resource group client, which causes the number of the allocated tokens to be less than the specified value #7346 @guo-shaoge
  • Fix the issue that the time data type in the INFORMATION_SCHEMA.RUNAWAY_WATCHES table is incorrect #54770 @HuSharp
  • Fix the issue that setting replication.strictly-match-label to true causes TiFlash to fail to start #8480 @rleungx

• TiFlash

  • Fix the issue that TiFlash write nodes might fail to restart in the disaggregated storage and compute architecture #9282 @JaySon-Huang
  • Fix the issue that a network partition (network disconnection) between TiFlash and any PD might cause read request timeout errors #9243 @Lloyd-Pottiger
  • Fix the issue that when using the CAST() function to convert a string to a datetime with a time zone or invalid characters, the result is incorrect #8754 @solotzg
  • Fix the issue that read snapshots of TiFlash write nodes are not released in a timely manner in the disaggregated storage and compute architecture #9298 @JinheLin
  • Fix the issue that TiFlash fails to parse the table schema when the table contains Bit-type columns with a default value that contains invalid characters #9461 @Lloyd-Pottiger
  • Fix the issue that some queries might report errors when late materialization is enabled #9472 @Lloyd-Pottiger
  • Fix the issue that converting a data type to the DECIMAL type might lead to wrong query results in some extreme cases #53892 @guo-shaoge

• Tools

  • Backup & Restore (BR)

    • Fix the issue that backup tasks might get stuck if TiKV becomes unresponsive during the backup process #53480 @Leavrth
    • Fix the issue that the checkpoint path of backup and restore is incompatible with some external storage #55265 @Leavrth
    • Fix the issue that after a log backup PITR task fails and you stop it, the safepoints related to that task are not properly cleared in PD #17316 @Leavrth
    • Fix the issue that BR logs might print sensitive credential information when log backup is enabled #55273 @RidRisR
    • Fix the issue that BR integration test cases are unstable, and add a new test case to simulate snapshot or log backup file corruption #53835 @Leavrth

  • TiCDC

    • Fix the issue that the barrier-ts monitoring metric for changefeed checkpoints might be inaccurate #11553 @3AceShowHand

  • TiDB Data Migration (DM)

    • Fix the issue that data replication is interrupted when the index length exceeds the default value of max-index-length #11459 @michaelmdeng
    • Fix the issue that DM does not set the default database when processing the ALTER DATABASE statement, which causes a replication error #11503 @lance6716
    • Fix the issue that multiple DM-master nodes might simultaneously become leaders, leading to data inconsistency #11602 @GMHDBJD

  • TiDB Lightning

    • Fix the issue that transaction conflicts occur during data import using TiDB Lightning #49826 @lance6716