连接参数
连接参数是在使用 CREATE CONNECTION 创建可复用连接时提供的键值对。连接创建后，可以通过 CONNECTION = (CONNECTION_NAME = '<connection-name>') 在 stage、COPY 命令以及其他 SQL 功能中引用该连接。完整语法和用法请参见 CREATE CONNECTION。
有关不同存储类型的连接详情，请参见下表。
下表列出了访问 Amazon S3 类存储服务的连接参数：
-- Create a reusable connection for Amazon S3
CREATE CONNECTION my_s3_conn
STORAGE_TYPE = 's3'
ACCESS_KEY_ID = '<your-ak>'
SECRET_ACCESS_KEY = '<your-sk>';
-- Use the connection when creating a stage
CREATE STAGE my_s3_stage
URL = 's3://my-bucket'
CONNECTION = (CONNECTION_NAME = 'my_s3_conn');
-- Create a reusable connection for an S3-compatible service such as MinIO
CREATE CONNECTION my_minio_conn
STORAGE_TYPE = 's3'
ENDPOINT_URL = 'http://localhost:9000'
ACCESS_KEY_ID = 'ROOTUSER'
SECRET_ACCESS_KEY = 'CHANGEME123';
CREATE STAGE my_minio_stage
URL = 's3://lake'
CONNECTION = (CONNECTION_NAME = 'my_minio_conn');
要访问你的 Amazon S3 存储桶，也可以指定 AWS IAM role 和 external ID 进行身份验证。通过指定 AWS IAM role 和 external ID，你可以更细粒度地控制用户可以访问哪些 S3 存储桶。这意味着，如果某个 IAM role 仅被授予访问特定 S3 存储桶的权限，那么用户也只能访问这些存储桶。external ID 还可以通过提供额外的一层验证来进一步增强安全性。更多信息，请参见 https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html
下表列出了使用 AWS IAM role 身份验证访问 Amazon S3 存储服务的连接参数：
-- Create the connection using IAM role authentication
CREATE CONNECTION my_iam_conn
STORAGE_TYPE = 's3'
ROLE_ARN = 'arn:aws:iam::123456789012:role/my-role'
EXTERNAL_ID = 'my-external-id';
-- Reference the connection when creating a stage
CREATE STAGE my_iam_stage
URL = 's3://my-bucket'
CONNECTION = (CONNECTION_NAME = 'my_iam_conn');
下表列出了访问 Azure Blob Storage 的连接参数：
-- Create a connection for Azure Blob Storage
CREATE CONNECTION my_azure_conn
STORAGE_TYPE = 'azblob'
ACCOUNT_NAME = 'myaccount'
ACCOUNT_KEY = 'myaccountkey'
ENDPOINT_URL = 'https://<your-storage-account-name>.blob.core.windows.net';
-- Create a stage that uses the connection
CREATE STAGE my_azure_stage
URL = 'azblob://my-container'
CONNECTION = (CONNECTION_NAME = 'my_azure_conn');
下表列出了访问 Google Cloud Storage 的连接参数：
要获取 credential，你可以参考 Google 文档中的主题 Create a service account key 来创建并下载服务账户密钥文件。下载服务账户密钥文件后，可以通过以下命令将其转换为 base64 字符串：
base64 -i -o ~/Desktop/base64-encoded-key.txt
-- Create the connection with the base64-encoded credential
CREATE CONNECTION my_gcs_conn
STORAGE_TYPE = 'gcs'
CREDENTIAL = '<your-base64-encoded-credential>';
-- Use the connection when creating a stage
CREATE STAGE my_gcs_stage
URL = 'gcs://my-bucket'
CONNECTION = (CONNECTION_NAME = 'my_gcs_conn');
下表列出了访问 Alibaba Cloud OSS 的连接参数：
-- Create a connection for Alibaba Cloud OSS
CREATE CONNECTION my_oss_conn
STORAGE_TYPE = 'oss'
ACCESS_KEY_ID = '<your-ak>'
ACCESS_KEY_SECRET = '<your-sk>'
ENDPOINT_URL = 'https://<bucket-name>.<region-id>[-internal].aliyuncs.com';
-- Create a stage using the connection
CREATE STAGE my_oss_stage
URL = 'oss://my-bucket'
CONNECTION = (CONNECTION_NAME = 'my_oss_conn');
下表列出了访问 Tencent Cloud Object Storage (COS) 的连接参数：
-- Create a connection for Tencent COS
CREATE CONNECTION my_cos_conn
STORAGE_TYPE = 'cos'
SECRET_ID = '<your-secret-id>'
SECRET_KEY = '<your-secret-key>'
ENDPOINT_URL = '<your-endpoint-url>';
-- Create a stage that uses the connection
CREATE STAGE my_cos_stage
URL = 'cos://my-bucket'
CONNECTION = (CONNECTION_NAME = 'my_cos_conn');
下表列出了访问 Hugging Face 的连接参数：
-- Create a connection for Hugging Face
CREATE CONNECTION my_hf_conn
STORAGE_TYPE = 'hf'
REPO_TYPE = 'dataset'
REVISION = 'main';
-- Create a stage that uses the connection
CREATE STAGE my_huggingface_stage
URL = 'hf://opendal/huggingface-testdata/'
CONNECTION = (CONNECTION_NAME = 'my_hf_conn');