TiDB 8.3.0 Release Notes

Release date: August 22, 2024

TiDB version: 8.3.0

Quick access: Quick start

8.3.0 introduces the following key features and improvements:

Feature details

Performance

• The optimizer allows pushing the Projection operator down to the storage engine by default #51876 @yibin87

  Pushing the Projection operator down to the storage engine reduces data transfer between the compute engine and the storage engine, thereby improving SQL execution performance. This is particularly effective for queries containing JSON query functions or JSON value attribute functions. Starting from v8.3.0, TiDB enables the Projection operator pushdown feature by default, by changing the default value of the system variable controlling this feature, tidb_opt_projection_push_down, from OFF to ON. When this feature is enabled, the optimizer automatically pushes eligible JSON query functions and JSON value attribute functions down to the storage engine.

  For more information, see documentation.

• Optimize batch processing strategy for KV (key-value) requests #55206 @zyguan

  TiDB fetches data by sending KV requests to TiKV. Batching and processing KV requests in bulk can significantly improve execution performance. Before v8.3.0, the batching strategy in TiDB is less efficient. Starting from v8.3.0, TiDB introduces several more efficient batching strategies in addition to the existing one. You can configure different batching strategies using the tikv-client.batch-policy configuration item to accommodate various workloads.

  For more information, see documentation.

• TiFlash introduces HashAgg aggregation calculation modes to improve the performance for high NDV data #9196 @guo-shaoge

  Before v8.3.0, TiFlash has low aggregation calculation efficiency during the first stage of HashAgg aggregation when handling data with high NDV (number of distinct values). Starting from v8.3.0, TiFlash introduces multiple HashAgg aggregation calculation modes to improve the aggregation performance for different data characteristics. To choose a desired HashAgg aggregation calculation mode, you can configure the tiflash_hashagg_preaggregation_mode system variable.

  For more information, see documentation.

• Ignore unnecessary columns when collecting statistics #53567 @hi-rustin

  When the optimizer generates an execution plan, it only needs statistics for some columns, such as columns in the filter conditions, columns in the join keys, and columns used for aggregation. Starting from v8.3.0, TiDB continuously observes the historical records of the columns used in SQL statements. By default, TiDB only collects statistics for columns with indexes and columns that are observed to require statistics collection. This speeds up the collection of statistics and avoids unnecessary resource consumption.

  When you upgrade your cluster from a version earlier than v8.3.0 to v8.3.0 or later, TiDB retains the original behavior by default, that is, collecting statistics for all columns. To enable this feature, you need to manually set the system variable tidb_analyze_column_options to PREDICATE. For newly deployed clusters, this feature is enabled by default.

  For analytical systems with many random queries, you can set the system variable tidb_analyze_column_options to ALL to collect statistics for all columns, to ensure the performance of random queries. For other types of systems, it is recommended to keep the default setting (PREDICATE) of tidb_analyze_column_options to collect statistics for only necessary columns.

  For more information, see documentation.

• Improve the query performance of some system tables #50305 @tangenta

  In previous versions, querying system tables has poor performance when the cluster size becomes large and there are a large number of tables.

  In v8.0.0, query performance is optimized for the following four system tables:

  • INFORMATION_SCHEMA.TABLES
  • INFORMATION_SCHEMA.STATISTICS
  • INFORMATION_SCHEMA.KEY_COLUMN_USAGE
  • INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS

  In v8.3.0, the query performance is optimized for the following system tables, bringing a multi-fold performance improvement compared to v8.2.0.

  • INFORMATION_SCHEMA.CHECK_CONSTRAINTS
  • INFORMATION_SCHEMA.COLUMNS
  • INFORMATION_SCHEMA.PARTITIONS
  • INFORMATION_SCHEMA.SCHEMATA
  • INFORMATION_SCHEMA.SEQUENCES
  • INFORMATION_SCHEMA.TABLE_CONSTRAINTS
  • INFORMATION_SCHEMA.TIDB_CHECK_CONSTRAINTS
  • INFORMATION_SCHEMA.TiDB_INDEXES
  • INFORMATION_SCHEMA.TIDB_INDEX_USAGE
  • INFORMATION_SCHEMA.VIEWS

• Support partition pruning when partition expressions use the EXTRACT(YEAR_MONTH...) function to improve query performance #54209 @mjonss

  In previous versions, when partition expressions use the EXTRACT(YEAR_MONTH...) function, partition pruning is not supported, resulting in poor query performance. Starting from v8.3.0, partition pruning is supported when partition expressions use the EXTRACT(YEAR_MONTH...) function, which improves query performance.

  For more information, see documentation.

• Improve the performance of CREATE TABLE by 1.4 times, CREATE DATABASE by 2.1 times, and ADD COLUMN by 2 times #54436 @D3Hunter

  TiDB v8.0.0 introduces the system variable tidb_enable_fast_create_table to improve table creation performance in batch table creation scenarios. In v8.3.0, when submitting the DDL statements for table creation concurrently through 10 sessions in a single database, the performance is improved by 1.4 times compared with v8.2.0.

  In v8.3.0, the performance of general DDLs in batch execution has improved compared to v8.2.0. The performance of CREATE DATABASE for 10 sessions concurrently improves by 19 times compared with v8.1.0 and 2.1 times compared with v8.2.0. The performance of using 10 sessions to add columns (ADD COLUMN) to multiple tables in the same database in batch has improved by 10 times compared with v8.1.0, and 2.1 times compared with v8.2.0. The performance of ADD COLUMN with 10 sessions on multiple tables in the same database has improved by 10 times compared with v8.1.0 and 2 times compared with v8.2.0.

  For more information, see documentation.

• Partitioned tables support global indexes (experimental) #45133 @mjonss @Defined2014 @jiyfhust @L-maple

  In previous versions of partitioned tables, some limitations exist because global indexes are not supported. For example, the unique key must use every column in the table's partitioning expression. If the query condition does not use the partitioning key, the query will scan all partitions, resulting in poor performance. Starting from v7.6.0, the system variable tidb_enable_global_index is introduced to enable the global index feature. But this feature was under development at that time and it is not recommended to enable it.

  Starting with v8.3.0, the global index feature is released as an experimental feature. You can explicitly create a global index for a partitioned table with the keyword Global to remove the restriction that the unique key must use every column in the table's partitioning expression, to meet flexible business needs. Global indexes also enhance the performance of queries that do not include partition keys.

  For more information, see documentation.

Reliability

• Support streaming cursor result sets (experimental) #54526 @YangKeao

  When the application code retrieves the result set using Cursor Fetch, TiDB usually first stores the complete result set in memory, and then returns the data to the client in batches. If the result set is too large, TiDB might temporarily write the result to the hard disk.

  Starting from v8.3.0, if you set the system variable tidb_enable_lazy_cursor_fetch to ON, TiDB no longer reads all data to the TiDB node, but gradually reads data to the TiDB node as the client reads. When TiDB processes large result sets, this feature reduces the memory usage of the TiDB node and improves the stability of the cluster.

  For more information, see documentation.

• Enhance SQL execution plan binding #55280 #55343 @time-and-fate

  In OLTP scenarios, the optimal execution plan for most SQL statements is fixed. Implementing SQL execution plan binding for important SQL statements in the application can reduce the probability of the execution plan becoming worse and improve system stability. To meet the requirements of creating a large number of SQL execution plan bindings, TiDB enhances the capability and experience of SQL binding, including:

  • Use a single SQL statement to create SQL execution plan bindings from multiple historical execution plans to improve the efficiency of creating bindings.
  • The SQL execution plan binding supports more optimizer hints, and optimizes the conversion method for complex execution plans, making the binding more stable in restoring the execution plan.

  For more information, see documentation.

Availability

• TiProxy supports built-in virtual IP management #583 @djshow832

  Before v8.3.0, when using primary-secondary mode for high availability, TiProxy requires an additional component to manage the virtual IP address. Starting from v8.3.0, TiProxy supports built-in virtual IP management. In primary-secondary mode, when a primary node fails over, the new primary node will automatically bind to the specified virtual IP, ensuring that clients can always connect to an available TiProxy through the virtual IP.

  To enable virtual IP management, specify the virtual IP address using the TiProxy configuration item ha.virtual-ip and specify the network interface to bind the virtual IP to using ha.interface. The virtual IP will be bound to a TiProxy instance only when both of these configuration items are set.

  For more information, see documentation.

SQL

• Support upgrading SELECT LOCK IN SHARE MODE to exclusive locks #54999 @cfzjywxk

  TiDB does not support SELECT LOCK IN SHARE MODE yet. Starting from v8.3.0, TiDB supports upgrading SELECT LOCK IN SHARE MODE to exclusive locks to enable support for SELECT LOCK IN SHARE MODE. You can control whether to enable this feature by using the new system variable tidb_enable_shared_lock_promotion.

  For more information, see documentation.

Observability

• Show the progress of loading initial statistics #53564 @hawkingrei

  TiDB loads basic statistics when it starts. In scenarios with many tables or partitions, this process can take a long time. When the configuration item force-init-stats is set to ON, TiDB does not provide services until the initial statistics are loaded. In this case, you need to observe the loading process to estimate the service start time.

  Starting from v8.3.0, TiDB prints the progress of loading initial statistics in stages in the log, so you can understand the running status. To provide formatted results to external tools, TiDB adds the additional monitoring API so you can obtain the progress of loading initial statistics at any time during the startup phase.

• Add metrics about Request Unit (RU) settings #8444 @nolouch

Security

• Enhance PD log redaction #8305 @JmPotato

  TiDB v8.0.0 enhances log redaction and supports marking user data in TiDB logs with ‹ ›. Based on the marked logs, you can decide whether to redact the marked information when displaying the logs, thus increasing the flexibility of log redaction. In v8.2.0, TiFlash implements a similar log redaction enhancement.

  In v8.3.0, PD implements a similar log redaction enhancement. To use this feature, you can set the value of the PD configuration item security.redact-info-log to "marker".

  For more information, see documentation.

• Enhance TiKV log redaction #17206 @lucasliang

  TiDB v8.0.0 enhances log redaction and supports marking user data in TiDB logs with ‹ ›. Based on the marked logs, you can decide whether to redact the marked information when displaying the logs, thus increasing the flexibility of log redaction. In v8.2.0, TiFlash implements a similar log redaction enhancement.

  In v8.3.0, TiKV implements a similar log redaction enhancement. To use this feature, you can set the value of the TiKV configuration item security.redact-info-log to "marker".

  For more information, see documentation.

Data migration

• TiCDC supports replicating DDL statements in bi-directional replication (BDR) mode (GA) #10301 #48519 @okJiang @asddongmen

  TiCDC v7.6.0 introduced the replication of DDL statements with bi-directional replication configured. Previously, bi-directional replication of DDL statements was not supported by TiCDC, so users of TiCDC's bi-directional replication had to execute DDL statements on both TiDB clusters separately. With this feature, after assigning a PRIMARY BDR role to a cluster, TiCDC can replicate the DDL statements from that cluster to the SECONDARY cluster.

  In v8.3.0, this feature becomes generally available (GA).

  For more information, see documentation.

Compatibility changes

Behavior changes

• To avoid incorrect use of commands, pd-ctl cancels the prefix matching mechanism. For example, store remove-tombstone cannot be called via store remove #8413 @lhy1024

System variables

Configuration file parameters

System tables

• The INFORMATION_SCHEMA.PROCESSLIST and INFORMATION_SCHEMA.CLUSTER_PROCESSLIST system tables add the SESSION_ALIAS field to show the number of rows currently affected by the DML statement #46889 @lcwangchao

Deprecated features

• The following features are deprecated starting from v8.3.0:

  • Starting from v7.5.0, TiDB Binlog replication is deprecated. Starting from v8.3.0, TiDB Binlog is fully deprecated, with removal planned for a future release. For incremental data replication, use TiCDC instead. For point-in-time recovery (PITR), use PITR.
  • Starting from v8.3.0, the tidb_enable_column_tracking system variable is deprecated. TiDB tracks predicate columns by default. For more information, see tidb_analyze_column_options.

• The following features are planned for deprecation in future versions:

  • TiDB introduces the system variable tidb_enable_auto_analyze_priority_queue, which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated.
  • TiDB introduces the system variable tidb_enable_async_merge_global_stats in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated.
  • It is planned to redesign the automatic evolution of execution plan bindings in subsequent releases, and the related variables and behavior will change.
  • In v8.0.0, TiDB introduces the tidb_enable_parallel_hashagg_spill system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the tidb_enable_parallel_hashagg_spill system variable will be deprecated.
  • The TiDB Lightning parameter conflict.max-record-rows is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by conflict.threshold, which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task.

• The following features are planned for removal in future versions:

  • Starting from v8.0.0, TiDB Lightning deprecates the old version of conflict detection strategy for the physical import mode, and enables you to control the conflict detection strategy for both logical and physical import modes via the conflict.strategy parameter. The duplicate-resolution parameter for the old version of conflict detection will be removed in a future release.

Improvements

• TiDB

  • Support the SELECT ... STRAIGHT_JOIN ... USING ( ... ) statement #54162 @dveeden
  • Construct more precise index access ranges for filter conditions like ((idx_col_1 > 1) or (idx_col_1 = 1 and idx_col_2 > 10)) and ((idx_col_1 < 10) or (idx_col_1 = 10 and idx_col_2 < 20)) #54337 @ghazalfamilyusa
  • Use index order to avoid extra sorting operations for SQL queries like WHERE idx_col_1 IS NULL ORDER BY idx_col_2 #54188 @ari-e
  • Display analyzed indexes in the mysql.analyze_jobs system table #53567 @hi-rustin
  • Support applying the tidb_redact_log setting to the output of EXPLAIN statements and further optimize the logic in processing logs #54565 @hawkingrei
  • Support generating the Selection operator on IndexRangeScan for multi-valued indexes to improve query efficiency #54876 @time-and-fate
  • Support killing automatic ANALYZE tasks that are running outside the set time window #55283 @hawkingrei
  • Adjust estimation results from 0 to 1 for equality conditions that do not hit TopN when statistics are entirely composed of TopN and the modified row count in the corresponding table statistics is non-zero #47400 @terry1purcell
  • The TopN operator supports disk spill #47733 @xzhangxian1008
  • TiDB node supports executing queries with the WITH ROLLUP modifier and the GROUPING function #42631 @Arenatlx
  • The system variable tidb_low_resolution_tso supports the GLOBAL scope #55022 @cfzjywxk
  • Improve GC (Garbage Collection) efficiency by supporting concurrent range deletion. You can control the number of concurrent threads using tidb_gc_concurrency #54570 @ekexium
  • Improve the performance of bulk DML execution mode (tidb_dml_type = "bulk") #50215 @ekexium
  • Improve the performance of schema information cache-related interface SchemaByID #54074 @ywqzzy
  • Improve the query performance for certain system tables when schema information caching is enabled #50305 @tangenta
  • Optimize error messages for conflicting keys when adding unique indexes #53004 @lance6716

• PD

  • Support modifying the batch configuration of the evict-leader-scheduler via pd-ctl to accelerate the leader eviction process #8265 @rleungx
  • Add the store_id monitoring metric to the Cluster > Label distribution panel in Grafana to display store IDs corresponding to different labels #8337 @HuSharp
  • Support fallback to the default resource group when the specified resource group does not exist #8388 @JmPotato
  • Add the approximate_kv_size field to the Region information output by the region command in pd-ctl #8412 @zeminzhou
  • Optimize the message that returns when you call the PD API to delete the TTL configuration #8450 @lhy1024
  • Optimize the RU consumption behavior of large query read requests to reduce the impact on other requests #8457 @nolouch
  • Optimize the error message that returns when you misconfigure PD microservices #52912 @rleungx
  • Add the --name startup parameter to PD microservices to more accurately display the service name during deployment #7995 @HuSharp
  • Support dynamically adjusting PatrolRegionScanLimit based on the number of Regions to reduce Region scan time #7963 @lhy1024

• TiKV

  • Optimize the batching policy for writing Raft logs when async-io is enabled to reduce the consumption of disk I/O bandwidth resources #16907 @LykxSassinator
  • Redesign the TiCDC delegate and downstream modules to better support Region partial subscription #16362 @hicqu
  • Reduce the size of a single slow query log #17294 @Connor1996
  • Add a new monitoring metric min safe ts #17307 @mittalrishabh
  • Reduce the memory usage of the peer message channel #16229 @Connor1996

• TiFlash

  • Support generating ad hoc heap profiling in SVG format #9320 @CalvinNeo

• Tools

  • Backup & Restore (BR)

    • Support checking whether a full backup exists before starting point-in-time recovery (PITR) for the first time. If the full backup is not found, BR terminates the restore and returns an error #54418 @Leavrth
    • Support checking whether the disk space in TiKV and TiFlash is sufficient before restoring snapshot backups. If the space is insufficient, BR terminates the restore and returns an error #54316 @RidRisR
    • Support checking whether the disk space in TiKV is sufficient before TiKV downloads each SST file. If the space is insufficient, BR terminates the restore and returns an error #17224 @RidRisR
    • Support setting Alibaba Cloud access credentials through environment variables #45551 @RidRisR
    • Optimize the backup feature, improving backup performance and stability during node restarts, cluster scaling-out, and network jitter when backing up large numbers of tables #52534 @3pointer
    • Automatically set the environment variable GOMEMLIMIT based on the available memory of the BR process to avoid OOM when using BR for backup and restore #53777 @Leavrth
    • Make incremental backups compatible with point-in-time recovery (PITR) #54474 @3pointer
    • Support backing up and restoring the mysql.column_stats_usage table #53567 @hi-rustin

Bug fixes

• TiDB

  • Reset the parameters in the Open method of PipelinedWindow to fix the unexpected error that occurs when the PipelinedWindow is used as a child node of Apply due to the reuse of previous parameter values caused by repeated opening and closing operations #53600 @XuHuaiyu
  • Fix the issue that the query might get stuck when terminated because the memory usage exceeds the limit set by tidb_mem_quota_query #55042 @yibin87
  • Fix the issue that the disk spill for the HashAgg operator causes incorrect query results during parallel calculation #55290 @xzhangxian1008
  • Fix the issue of wrong JSON_TYPE when casting YEAR to JSON format #54494 @YangKeao
  • Fix the issue that the value range of the tidb_schema_cache_size system variable is wrong #54034 @lilinghai
  • Fix the issue that partition pruning does not work when the partition expression is EXTRACT(YEAR FROM col) #54210 @mjonss
  • Fix the issue that FLASHBACK DATABASE fails when many tables exist in the database #54415 @lance6716
  • Fix the issue that FLASHBACK DATABASE enters an infinite loop when handling many databases #54915 @lance6716
  • Fix the issue that adding an index in index acceleration mode might fail #54568 @lance6716
  • Fix the issue that ADMIN CANCEL DDL JOBS might cause DDL to fail #54687 @lance6716
  • Fix the issue that table replication fails when the index length of the table replicated from DM exceeds the maximum length specified by max-index-length #55138 @lance6716
  • Fix the issue that the error runtime error: index out of range might occur when executing SQL statements with tidb_enable_inl_join_inner_multi_pattern enabled #54535 @joechenrh
  • Fix the issue that you cannot exit TiDB using Control+C during the process of initializing statistics #54589 @tiancaiamao
  • Fix the issue that the INL_MERGE_JOIN optimizer hint returns incorrect results by deprecating it #54064 @AilinKid
  • Fix the issue that a correlated subquery that contains WITH ROLLUP might cause TiDB to panic and return the error runtime error: index out of range #54983 @AilinKid
  • Fix the issue that predicates cannot be pushed down properly when the filter condition of a SQL query contains virtual columns and the execution condition contains UnionScan #54870 @qw4990
  • Fix the issue that the error runtime error: invalid memory address or nil pointer dereference might occur when executing SQL statements with tidb_enable_inl_join_inner_multi_pattern enabled #55169 @hawkingrei
  • Fix the issue that a query statement that contains UNION might return incorrect results #52985 @XuHuaiyu
  • Fix the issue that the tot_col_size column in the mysql.stats_histograms table might be a negative number #55126 @qw4990
  • Fix the issue that columnEvaluator cannot identify the column references in the input chunk, which leads to runtime error: index out of range when executing SQL statements #53713 @AilinKid
  • Fix the issue that STATS_EXTENDED becomes a reserved keyword #39573 @wddevries
  • Fix the issue that when tidb_low_resolution is enabled, select for update can be executed #54684 @cfzjywxk
  • Fix the issue that internal SQL queries cannot be displayed in the slow query log when tidb_redact_log is enabled #54190 @lcwangchao
  • Fix the issue that the memory used by transactions might be tracked multiple times #53984 @ekexium
  • Fix the issue that using SHOW WARNINGS; to obtain warnings might cause a panic #48756 @xhebox
  • Fix the issue that loading index statistics might cause memory leaks #54022 @hi-rustin
  • Fix the issue that the LENGTH() condition is unexpectedly removed when the collation is utf8_bin or utf8mb4_bin #53730 @elsa0520
  • Fix the issue that statistics collection does not update the stats_history table when encountering duplicate primary keys #47539 @Defined2014
  • Fix the issue that recursive CTE queries might result in invalid pointers #54449 @hawkingrei
  • Fix the issue that the Connection Count monitoring metric in Grafana is incorrect when some connections exit before the handshake is complete #54428 @YangKeao
  • Fix the issue that the Connection Count of each resource group is incorrect when using TiProxy and resource groups #54545 @YangKeao
  • Fix the issue that when queries contain non-correlated subqueries and LIMIT clauses, column pruning might be incomplete, resulting in a less optimal plan #54213 @qw4990
  • Fix the issue of reusing wrong point get plans for SELECT ... FOR UPDATE #54652 @qw4990
  • Fix the issue that the TIMESTAMPADD() function goes into an infinite loop when the first argument is month and the second argument is negative #54908 @xzhangxian1008
  • Fix the issue that internal SQL statements in the slow log are redacted to null by default #54190 #52743 #53264 @lcwangchao
  • Fix the issue that PointGet execution plans for _tidb_rowid can be generated #54583 @Defined2014
  • Fix the issue that SHOW IMPORT JOBS reports an error Unknown column 'summary' after upgrading from v7.1 #54241 @tangenta
  • Fix the issue that obtaining the column information using information_schema.columns returns warning 1356 when a subquery is used as a column definition in a view definition #54343 @lance6716
  • Fix the issue that RANGE partitioned tables that are not strictly self-incrementing can be created #54829 @Defined2014
  • Fix the issue that INDEX_HASH_JOIN cannot exit properly when SQL is abnormally interrupted #54688 @wshwsh12
  • Fix the issue that the network partition during adding indexes using the Distributed eXecution Framework (DXF) might cause inconsistent data indexes #54897 @tangenta

• PD

  • Fix the issue that no error is reported when binding a role to a resource group #54417 @JmPotato
  • Fix the issue that a resource group encounters quota limits when requesting tokens for more than 500 ms #8349 @nolouch
  • Fix the issue that the time data type in the INFORMATION_SCHEMA.RUNAWAY_WATCHES table is incorrect #54770 @HuSharp
  • Fix the issue that resource groups could not effectively limit resource usage under high concurrency #8435 @nolouch
  • Fix the issue that an incorrect PD API is called when you retrieve table attributes #55188 @JmPotato
  • Fix the issue that the scaling progress is displayed incorrectly after the scheduling microservice is enabled #8331 @rleungx
  • Fix the issue that the encryption manager is not initialized before use #8384 @rleungx
  • Fix the issue that some logs are not redacted #8419 @rleungx
  • Fix the issue that redirection might panic during the startup of PD microservices #8406 @HuSharp
  • Fix the issue that the split-merge-interval configuration item might not take effect when you modify its value repeatedly (such as changing it from 1s to 1h and back to 1s) #8404 @lhy1024
  • Fix the issue that setting replication.strictly-match-label to true causes TiFlash to fail to start #8480 @rleungx
  • Fix the issue that fetching TSO is slow when analyzing large partitioned tables, causing ANALYZE performance degradation #8500 @rleungx
  • Fix the potential data races in large clusters #8386 @rleungx
  • Fix the issue that when determining whether queries are Runaway Queries, TiDB only counts time consumption spent on the Coprocessor side while missing time consumption spent on the TiDB side, resulting in some queries not being identified as Runaway Queries #51325 @HuSharp

• TiFlash

  • Fix the issue that when using the CAST() function to convert a string to a datetime with a time zone or invalid characters, the result is incorrect #8754 @solotzg
  • Fix the issue that TiFlash might panic after executing RENAME TABLE ... TO ... on a partitioned table with empty partitions across databases #9132 @JaySon-Huang
  • Fix the issue that some queries might report a column type mismatch error after late materialization is enabled #9175 @JinheLin
  • Fix the issue that queries with virtual generated columns might return incorrect results after late materialization is enabled #9188 @JinheLin
  • Fix the issue that setting the SSL certificate configuration to an empty string in TiFlash incorrectly enables TLS and causes TiFlash to fail to start #9235 @JaySon-Huang
  • Fix the issue that TiFlash might panic when a database is deleted shortly after creation #9266 @JaySon-Huang
  • Fix the issue that a network partition (network disconnection) between TiFlash and any PD might cause read request timeout errors #9243 @Lloyd-Pottiger
  • Fix the issue that TiFlash write nodes might fail to restart in the disaggregated storage and compute architecture #9282 @JaySon-Huang
  • Fix the issue that read snapshots of TiFlash write nodes are not released in a timely manner in the disaggregated storage and compute architecture #9298 @JinheLin

• TiKV

  • Fix the issue that cleaning up stale regions might accidentally delete valid data #17258 @hbisheng
  • Fix the issue that Ingestion picked level and Compaction Job Size(files) are displayed incorrectly in the TiKV dashboard in Grafana #15990 @Connor1996
  • Fix the issue that cancel_generating_snap incorrectly updating snap_tried_cnt causes TiKV to panic #17226 @hbisheng
  • Fix the issue that the information of Ingest SST duration seconds is incorrect #17239 @LykxSassinator
  • Fix the issue that CPU profiling flag is not reset correctly when an error occurs #17234 @Connor1996
  • Fix the issue that bloom filters are incompatible between earlier versions (earlier than v7.1) and later versions #17272 @v01dstar

• Tools

  • Backup & Restore (BR)

    • Fix the issue that DDLs requiring backfilling, such as ADD INDEX and MODIFY COLUMN, might not be correctly recovered during incremental restore #54426 @3pointer
    • Fix the issue that the progress is stuck during backup and restore #54140 @Leavrth
    • Fix the issue that the checkpoint path of backup and restore is incompatible with some external storage #55265 @Leavrth

  • TiCDC

    • Fix the issue that the processor might get stuck when the downstream Kafka is inaccessible #11340 @asddongmen

  • TiDB Data Migration (DM)

    • Fix the issue that schema tracker incorrectly handles LIST partition tables, causing DM errors #11408 @lance6716
    • Fix the issue that data replication is interrupted when the index length exceeds the default value of max-index-length #11459 @michaelmdeng
    • Fix the issue that DM cannot handle FAKE_ROTATE_EVENT correctly #11381 @lance6716

  • TiDB Lightning

    • Fix the issue that TiDB Lightning outputs a confusing WARN log when it fails to obtain the keyspace name #54232 @kennytm
    • Fix the issue that the TLS configuration of TiDB Lightning affects cluster certificates #54172 @ei-sugimoto
    • Fix the issue that transaction conflicts occur during data import using TiDB Lightning #49826 @lance6716
    • Fix the issue that large checkpoint files cause performance degradation during the import of numerous databases and tables #55054 @D3Hunter

Contributors

We would like to thank the following contributors from the TiDB community:

• ari-e
• ei-sugimoto
• HaoW30
• JackL9u
• michaelmdeng
• mittalrishabh
• qingfeng777
• renovate
• SandeepPadhi
• yzhan1