Database Audit Logging

TiDB Cloud provides you with a database audit logging feature to record a history of user access details (such as any SQL statements executed) in logs.

To assess the effectiveness of user access policies and other information security measures of your organization, it is a security best practice to conduct a periodic analysis of the database audit logs.

The audit logging feature is disabled by default. To audit a cluster, you need to enable the audit logging first, and then specify the auditing filter rules.

Prerequisites

• You are using a TiDB Cloud Dedicated cluster. Audit logging is not available for TiDB Cloud Serverless clusters.
• You are in the Organization Owner or Project Owner role of your organization. Otherwise, you cannot see the database audit-related options in the TiDB Cloud console. For more information, see User roles.

Enable audit logging

TiDB Cloud supports recording the audit logs of a TiDB Cloud Dedicated cluster to your cloud storage service. Before enabling database audit logging, configure your cloud storage service on the cloud provider where the cluster is located.

Enable audit logging for AWS

To enable audit logging for AWS, take the following steps:

Step 1. Create an Amazon S3 bucket

Specify an Amazon S3 bucket in your corporate-owned AWS account as a destination to which TiDB Cloud writes the audit logs.

Note:

Do not enable object lock on the AWS S3 bucket. Enabling object lock will prevent TiDB Cloud from pushing audit log files to S3.

For more information, see Creating a bucket in the AWS User Guide.

Step 2. Configure Amazon S3 access

1. Get the TiDB Cloud Account ID and the External ID of the TiDB cluster that you want to enable audit logging.

   1. In the TiDB Cloud console, navigate to the Clusters page of your project.

      Tip

      If you have multiple projects, you can click in the lower-left corner and switch to another project.

   2. Click the name of your target cluster to go to its overview page, and then click DB Audit Logging in the left navigation pane.

   3. On the DB Audit Logging page, click Enable in the upper-right corner.

   4. In the Enable Database Audit Logging dialog, locate the AWS IAM Policy Settings section, and record TiDB Cloud Account ID and TiDB Cloud External ID for later use.

2. In the AWS Management Console, go to IAM > Access Management > Policies, and then check whether there is a storage bucket policy with the s3:PutObject write-only permission.

   • If yes, record the matched storage bucket policy for later use.

   • If not, go to IAM > Access Management > Policies > Create Policy, and define a bucket policy according to the following policy template.

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Effect": "Allow",
                 "Action": "s3:PutObject",
                 "Resource": "<Your S3 bucket ARN>/*"
             }
         ]
     }
     

     In the template, <Your S3 bucket ARN> is the Amazon Resource Name (ARN) of your S3 bucket where the audit log files are to be written. You can go to the Properties tab in your S3 bucket and get the ARN value in the Bucket Overview area. In the "Resource" field, you need to add /* after the ARN. For example, if the ARN is arn:aws:s3:::tidb-cloud-test, you need to configure the value of the "Resource" field as "arn:aws:s3:::tidb-cloud-test/*".

3. Go to IAM > Access Management > Roles, and then check whether a role whose trust entity corresponds to the TiDB Cloud Account ID and the External ID that you recorded earlier already exists.

   • If yes, record the matched role for later use.
   • If not, click Create role, select Another AWS account as the trust entity type, and then enter the TiDB Cloud Account ID value into the Account ID field. Then, choose the Require External ID option and enter the TiDB Cloud External ID value into the External ID field.

4. In IAM > Access Management > Roles, click the role name from the previous step to go to the Summary page, and then take the following steps:

   1. Under the Permissions tab, check whether the recorded policy with the s3:PutObject write-only permission is attached to the role. If not, choose Attach Policies, search for the needed policy, and then click Attach Policy.
   2. Return to the Summary page and copy the Role ARN value to your clipboard.

Step 3. Enable audit logging

In the TiDB Cloud console, go back to the Enable Database Audit Logging dialog box where you got the TiDB Cloud account ID and the External ID values, and then take the following steps:

1. In the Bucket URI field, enter the URI of your S3 bucket where the audit log files are to be written.

2. In the Bucket Region drop-down list, select the AWS region where the bucket locates.

3. In the Role ARN field, fill in the Role ARN value that you copied in Step 2. Configure Amazon S3 access.

4. Click Test Connection to verify whether TiDB Cloud can access and write to the bucket.

   If it is successful, The connection is successfully is displayed. Otherwise, check your access configuration.

5. Click Enable to enable audit logging for the cluster.

   TiDB Cloud is ready to write audit logs for the specified cluster to your Amazon S3 bucket.

Enable audit logging for Google Cloud

To enable audit logging for Google Cloud, take the following steps:

Step 1. Create a GCS bucket

Specify a Google Cloud Storage (GCS) bucket in your corporate-owned Google Cloud account as a destination to which TiDB Cloud writes audit logs.

For more information, see Creating storage buckets in the Google Cloud Storage documentation.

Step 2. Configure GCS access

1. Get the Google Cloud Service Account ID of the TiDB cluster that you want to enable audit logging.

   1. In the TiDB Cloud console, navigate to the Clusters page of your project.

      Tip

      If you have multiple projects, you can click in the lower-left corner and switch to another project.

   2. Click the name of your target cluster to go to its overview page, and then click DB Audit Logging in the left navigation pane.

   3. On the DB Audit Logging page, click Enable in the upper-right corner.

   4. In the Enable Database Audit Logging dialog, locate the Google Cloud Server Account ID section, and record Service Account ID for later use.

2. In the Google Cloud console, go to IAM & Admin > Roles, and then check whether a role with the following write-only permissions of the storage container exists.

   • storage.objects.create
   • storage.objects.delete

   If yes, record the matched role for the TiDB cluster for later use. If not, go to IAM & Admin > Roles > CREATE ROLE to define a role for the TiDB cluster.

3. Go to Cloud Storage > Browser, select the GCS bucket you want TiDB Cloud to access, and then click SHOW INFO PANEL.

   The panel is displayed.

4. In the panel, click ADD PRINCIPAL.

   The dialog box for adding principals is displayed.

5. In the dialog box, take the following steps:

   1. In the New Principals field, paste the Google Cloud Service Account ID of the TiDB cluster.
   2. In the Role drop-down list, choose the role of the target TiDB cluster.
   3. Click SAVE.

Step 3. Enable audit logging

In the TiDB Cloud console, go back to the Enable Database Audit Logging dialog box where you got the TiDB Cloud account ID, and then take the following steps:

1. In the Bucket URI field, enter your full GCS bucket name.

2. In the Bucket Region field, select the GCS region where the bucket locates.

3. Click Test Connection to verify whether TiDB Cloud can access and write to the bucket.

   If it is successful, The connection is successfully is displayed. Otherwise, check your access configuration.

4. Click Enable to enable audit logging for the cluster.

   TiDB Cloud is ready to write audit logs for the specified cluster to your GCS bucket.

Specify auditing filter rules

After enabling audit logging, you must specify auditing filter rules to control which user access events to capture and write to audit logs versus which events to ignore. If no filter rules are specified, TiDB Cloud does not log anything.

To specify auditing filter rules for a cluster, take the following steps:

1. In the Audit Logging dialog box where you enable audit logging, scroll down and locate the Filter Rules section.
2. Add one or more filter rules, one rule per row, with each rule specifying a user expression, database expression, table expression, and access type.

View audit logs

TiDB Cloud audit logs are readable text files with the cluster ID, Pod ID, and log creation date incorporated into the fully qualified filenames.

For example, 13796619446086334065/tidb-0/tidb-audit-2022-04-21T18-16-29.529.log. In this example, 13796619446086334065 indicates the cluster ID and tidb-0 indicates the Pod ID.

Disable audit logging

If you no longer want to audit a cluster, go to the page of the cluster, click Settings > Audit Settings, and then toggle the audit setting in the upper-right corner to Off.

Audit log fields

For each database event record in audit logs, TiDB provides the following fields:

Depending on the EVENT_CLASS field value set by TiDB, database event records in audit logs also contain additional fields as follows:

• If the EVENT_CLASS value is CONNECTION, database event records also contain the following fields:

  Col #	Field name	TiDB data type	Maximum length	Description
  17	CLIENT_PORT	INTEGER		Client port number
  18	CONNECTION_ID	INTEGER		Connection ID
  19	CONNECTION_TYPE	VARCHAR	12	Connection via socket or unix-socket
  20	SERVER_ID	INTEGER		TiDB server ID
  21	SERVER_PORT	INTEGER		The port that the TiDB server uses to listen to client communicating via the MySQL protocol
  22	SERVER_OS_LOGIN_USER	VARCHAR	17	The username of the TiDB process startup system
  23	OS_VERSION	VARCHAR	N/A	The version of the operating system where the TiDB server is located
  24	SSL_VERSION	VARCHAR	6	The current SSL version of TiDB
  25	PID	INTEGER		The PID of the TiDB process

• If the EVENT_CLASS value is TABLE_ACCESS or GENERAL, database event records also contain the following fields:

  Col #	Field name	TiDB data type	Maximum length	Description
  17	CONNECTION_ID	INTEGER		Connection ID
  18	COMMAND	VARCHAR	14	The command type of the MySQL protocol
  19	SQL_STATEMENT	VARCHAR	17	The SQL statement type
  20	PID	INTEGER		The PID of the TiDB process