Skip to main content

File

Overview​

This Destination type can be used to sync data as a delimited file into your Amazon S3 or Azure Blob Storage repositories.

info

Before you set up your data sync destination, make sure to configure your Source.

tip

The File destination supports batch and real-time syncs.

Destination tab​

The following table outlines the mandatory and optional parameters you will find on the Destination tab.

The following parameters will help to define your data sync destination and how it functions.

ParameterDescriptionExample
DestinationMandatory. Select your destination from the drop down menu.File
SourceMandatory. Select whether your file is hosted in Amazon S3 or in Azure Blob Storage. Each source type contains unique parameters, detailed below.Amazon S3
Region (Amazon S3)Mandatory when using Amazon S3. Select the region where your Amazon S3 bucket is hosted.Canada (Central)
Bucket (Amazon S3)Mandatory when using Amazon S3. The name of the bucket that contains your file, as it appears in Amazon.my-bucket
Auth Type (Amazon S3)Mandatory when using Amazon S3. Select what type of authorization you will use to access your S3 bucket and file. This value will either be Access Key or IAM Role.Access Key
Access Key ID (Amazon S3)Mandatory when using an S3 access key. The Access Key ID for an account with the permissions to the above S3 bucket and file. The Connections UI will encrypt this value upon entry.AKIA0123456787EXAMPLE
Secret Access Key (Amazon S3)Mandatory when using an S3 access key. The Secret Key associated with the above Access Key ID. The Connections UI will encrypt this value upon entry.wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
FilenameMandatory. The name of the file you want to sync your data into.my-file
Container (Azure Blob Storage)Mandatory when using Azure Blob Storage. The name of the container that contains your file, as it appears in Azure.my-container
Auth Type (Azure Blob Storage)Mandatory when using Azure Blob Storage. Select what type of authorization you will use to access your container and file. This value will either be Connection String or Registered Application.Connection String
Connection String (Azure Blob Storage)Mandatory when using an Azure Blob Storage connection string authentication method. The connection string to access the above container and file. The Connections UI will encrypt this value upon entry.DefaultEndpointsProtocol=https;
AccountName=storagesample;AccountKey=account-key
Service URL (Azure Blob Storage)Mandatory when using Azure Blob Storage Registered Application authentication method. This is the URL of your storage account.https://mystorageaccount.blob.core.windows.
net/mycontainer/myblob
Tenant ID (Azure Blob Storage)Mandatory when using Azure Blob Storage Registered Application authentication method. The Tenant ID for the registered application that will access the container and file. The Connections UI will encrypt this value upon entry.abcde-1234-5678-fghij
Client ID (Azure Blob Storage)Mandatory when using Azure Blob Storage Registered Application authentication method. The Client ID for the registered application that will access the container and file. The Connections UI will encrypt this value upon entry. (Also known as the "Application ID")klmno-98765-pqrst
Client Secret (Azure Blob Storage)Mandatory when using Azure Blob Storage Registered Application authentication method. The Client Secret for the registered application that will access the container and file. The Connections UI will encrypt this value upon entry.uvwxyz-54321-abcde
File TypeThe type of file you are using as a destination. Currently this value only accepts JSON.JSON
Maximum Rows per FileThis value sets a row limit when syncing into your file. The numerical value must be greater than 0 and less/equal to 2,147,483,647.100
Test ConnectionYou can use the "Test Connection" button to ensure that your credentials are properly configured to access your destination. If configured correctly, a "Connection Successful" pop-up will appear. If configured incorrectly, a "Connection Failed" pop-up will appear along with a link to the applicable error logs to help you troubleshoot.
info

The Maximum Rows Per File parameter defines how many rows will be saved per file. If multiple files are required, they will be appended with a 9 digit suffix with leading zeros. For example: If the filename entered by the user is "MyFile.json", the files saved in the blob container will have names "MyFile_000000001.json", "MyFile_000000002.json", etc.

Current default encoding for saved JSON files is UTF-8.

Next steps​