Explore

5X S3 Data Lake Ingestion Setup Guide

This guide will help you connect your Amazon S3 data lake to the 5X ingestion platform for seamless data integration.

Setup Instructions

1. Important: External ID

For this integration, use the following External ID: reorder_sequence

This ID will be needed when creating the IAM role for 5X.

2. Create IAM Policy for S3 Bucket

Open your Amazon IAM console

Go to Policies, and then click Create Policy

⁠

Click the JSON tab

⁠

Copy the following policy and paste it in the JSON editor:

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "AllowListBucketOfASpecificPrefix",

"Effect": "Allow",

"Action": [

"s3:ListBucket"

"Resource": [

"arn:aws:s3:::{your-bucket-name}"

"Condition": {

"StringLike": {

"s3:prefix": [

"{prefix_path}/*"

]

}

{

"Sid": "AllowAllObjectActionsInSpecificPrefix",

"Effect": "Allow",

"Action": [

"s3:DeleteObjectTagging",

"s3:ReplicateObject",

"s3:PutObject",

"s3:GetObjectAcl",

"s3:GetObject",

"s3:DeleteObjectVersion",

"s3:PutObjectTagging",

"s3:DeleteObject",

"s3:PutObjectAcl"

"Resource": [

"arn:aws:s3:::{your-bucket-name}/{prefix_path}/*"

]

}

]

}

In the policy, replace:

{your-bucket-name} with the name of your S3 bucket

{prefix_path} with the prefix path of your S3 bucket

NOTE: Setting the "s3:prefix" condition to ["*"] grants access to all prefixes within the specified bucket, while setting it to ["{prefix_path}/*"] restricts access to a specific prefix path within the bucket.

Click Next

In the Policy name field, enter a name for your policy (e.g., "5X-S3-Access-Policy"), and then click Create policy

3. Create IAM Role

In the IAM console, go to Roles, and then click Create role

⁠

Select AWS account, and then select Another AWS account

In the Account ID field, enter 834469178297 (5X's account ID)

⁠

Select the Require external ID checkbox, and then enter the External ID: reorder_sequence

⁠

Click Next

When prompted to add permissions to the role, select the IAM Policy that was created in Step 2 and attach it to this role.

⁠

Click Next

In the Role name field, enter a name for the role (e.g., "5X-S3-Access-Role"), and then click Create role

In the Roles page, select the role you created

Make a note of the ARN. You will need to share this with 5X to configure your ingestion setup

4. Information to Share with 5X

After completing the setup, please provide the following information to your 5X support engineer:

S3 Bucket name

Fivetran Role ARN (the IAM role ARN you created)

S3 Prefix Path (optional, if you specified one)

S3 Bucket Region

Additional Information

We will be using Delta Lake (Parquet) as the table format for your data

Our data processing region is set to US for your integration

For optimal performance, we recommend that your S3 bucket is also located in a US region

If you have any questions or need assistance during the setup process, please don’t hesitate to reach out to us.

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.