Schema Cache

In some multi-tenant scenarios, there might be hundreds of thousands or even millions of databases and tables. Loading the schema information of all these databases and tables into memory would not only consume a large amount of memory but also degrade access performance. To address this issue, TiDB introduces a schema caching mechanism similar to LRU (Least Recently Used). Only the schema information of the most recently accessed databases and tables is cached in memory.

Configure schema cache

You can enable the schema caching feature by configuring the system variable tidb_schema_cache_size.

Best practices

• In scenarios with a large number of databases and tables (for example, more than 100,000 databases and tables) or when the number of databases and tables is large enough to affect system performance, it is recommended to enable the schema caching feature.
• You can monitor the hit rate of the schema cache by observing the subpanel Infoschema v2 Cache Operation under the Schema load section in TiDB Dashboard. If the hit rate is low, you can increase the value of tidb_schema_cache_size.
• You can monitor the current size of the schema cache being used by observing the subpanel Infoschema v2 Cache Size under the Schema load section in TiDB Dashboard.
• It is recommended to disable performance.force-init-stats to reduce TiDB startup time.
• If you need to create a large number of tables (for example, more than 100,000 tables), it is recommended to set the split-table parameter to false to reduce the number of Regions and thus decrease TiKV's memory usage.

Known limitations

In scenarios with a large number of databases and tables, the following known issues exist:

• When the tables are irregularly accessed, such as one set of tables are accessed at time1 while another set are accessed at time2, and the value of tidb_schema_cache_size is small, the schema information might be frequently evicted and cached, leading to performance fluctuations. This feature is more suitable for scenarios where frequently accessed databases and tables are relatively fixed.

• Statistics information might not be collected in a timely manner.

• Access to some metadata information might become slower.

• Switching the schema cache on or off requires a waiting period.

• Operations that involve enumerating all metadata information might become slower, such as:

  • SHOW FULL TABLES
  • FLASHBACK
  • ALTER TABLE ... SET TIFLASH MODE ...